In short
Senior Data Engineer in Sydney designing, building, and supporting scalable batch and streaming data pipelines and lakehouse solutions on Azure Databricks, integrating Azure Data Lake Gen2, Data Factory, Key Vault, and DevOps, and leading production Databricks notebooks, jobs, and CI/CD workflows.
About the role
XPT Software is seeking a Senior Data Engineer in Sydney, Australia, to design, build, and support scalable data pipelines and lakehouse solutions using Azure Databricks and the Microsoft Azure data ecosystem.
You will design batch and streaming data pipelines, implement lakehouse patterns, integrate with Azure Data Lake Gen2, Data Factory, Key Vault, and DevOps, and lead production-grade Databricks notebooks, jobs, and CI/CD workflows.
Requirements
- Senior-level data engineering experience
- Design, build, and support scalable data pipelines and lakehouse solutions on Azure Databricks
- Design batch and streaming data pipelines
- Implement lakehouse patterns
- Integrate with Azure Data Lake Gen2
- Integrate with Azure Data Factory
- Integrate with Azure Key Vault
- Integrate with Azure DevOps
- Lead production-grade Databricks notebooks and jobs
- Lead CI/CD workflows