About the role
TP is hiring a Senior Data Engineer to design and optimise data pipelines on an Azure and Databricks lakehouse. You will own data ingestion, modelling and delivery across the business, collaborating with data scientists, BI developers and stakeholders.
You will build scalable ETL/ELT pipelines using Azure Data Factory, Databricks and Spark, manage Delta Lake tables, and drive data governance, quality and security. Strong SQL, Python and cloud skills are essential.
Design and build scalable ETL/ELT data pipelines with Azure Data Factory, Databricks and Apache Spark. Build and manage data lakes and Delta tables on Azure Data Lake Storage Gen2 and Delta Lake. Optimise Spark workloads for performance, scalability and cost efficiency. Embed data quality, data governance and security practices across pipelines. Partner with analytics teams to enable self-service reporting in Power BI and Power Apps. Monitor and troubleshoot production jobs using Azure Monitor and Databricks job runs. Mentor junior data engineers and set standards for coding, architecture and DevOps. 5+ years in data engineering or related field. Hands-on experience with Azure cloud services. Experience with Databricks, Spark, and Delta Lake. Advanced SQL and Python, with distributed data processing. CI/CD and IaC using Terraform, GitHub Actions, or Azure DevOps. Strong data modelling fundamentals. Excellent English communication for diverse audiences. Ownership and autonomy in distributed teams. Nice to have: Kafka, Azure certifications, MLOps, Purview/Unity Catalog, GDPR knowledge.