About the role
EY GDS seeks a Senior Data Engineer with Databricks, Azure, Python, and SQL expertise to drive data modernization and cloud transformation programs. You will design scalable data pipelines, Lakehouse architectures, and Azure-based data platforms while collaborating across Data, AI, Analytics, and Cloud teams.
You will mentor junior engineers, optimize Spark workloads, and contribute to DataOps practices, ensuring governance, quality, and security across global clients.
Design, build, and optimize data pipelines using Databricks, PySpark, Python, and SQL. Develop scalable ETL/ELT solutions and Lakehouse architectures. Implement solutions using Delta Lake, Unity Catalog, Databricks Workflows, and Structured Streaming. Develop and support Azure-based data platforms leveraging ADF and ADLS Gen2. Optimize Spark workloads for performance, scalability, and cost efficiency. Implement CI/CD, DataOps, and engineering best practices. Collaborate with business and technical stakeholders to deliver high-quality data solutions. Support data governance, quality, and security initiatives. Mentor junior engineers and contribute to capability development. 5+ years of experience in Data Engineering. Strong hands-on experience with Databricks. Advanced proficiency in Python and SQL. Experience developing scalable data pipelines using PySpark and Databricks. Experience with Unity Catalog, Databricks Workflows, and Structured Streaming. Experience with Azure Data Factory (ADF) and ADLS Gen2. Strong understanding of Data Modeling, ETL/ELT, and Data Warehousing concepts. Experience with CI/CD using Azure DevOps, GitHub, or Git. Experience optimizing Spark workloads in large-scale environments. Excellent communication, problem-solving, and stakeholder management skills.