About the role
GEP Worldwide is seeking a Senior Data Engineer to shape our Databricks lakehouse architecture and drive strategic data initiatives. You will lead the design and implementation of complex data solutions, set engineering standards, and mentor engineers across cross-functional teams.
The role emphasizes performance, governance, and scalable pipelines. You will collaborate with analysts, product managers, and stakeholders to translate business needs into robust data solutions, ensuring data quality
Lead design, development, and optimization of scalable data solutions on Databricks using Python, SQL, and PySpark. Architect lakehouse solutions with Delta Lake and medallion architecture. Define and govern data assets in Unity Catalog, including schemas and access control. Design and build batch and incremental pipelines with Lakeflow and MERGE/CDC patterns. Own CI/CD for Databricks solutions with DABs, Git, and automated testing. Mentor junior engineers and participate in technical interviews and reviews. Expert-level hands-on experience with Databricks, Lakeflow, and Delta Lake architectures. Deep understanding of Spark and distributed data processing including performance tuning. Expert-level Python and SQL skills for data engineering (PySpark, Pandas). Strong experience with Unity Catalog governance, access control, and data lineage. Experience with CI/CD for data solutions and automated testing (pytest). Knowledge of data modeling, lakehouse/medallion architecture and Delta Lake optimization.