In short
Senior Data Engineer building and optimizing enterprise data pipelines on Azure Databricks in Mexico City: designing ingestion, transformation, and reporting-ready datasets with Medallion architecture, ADLS, ADF, PostgreSQL, data quality controls, and CI/CD via Azure DevOps, working with architects, BI, QA, and analysts.
About the role
Papigen is seeking a Senior Data Engineer / Databricks Engineer to design, develop, and optimize enterprise data pipelines using Azure Databricks, ADLS, ADF, PostgreSQL, and Azure DevOps in Mexico City. The role focuses on building ingestion, transformation, and reporting-ready datasets, implementing Medallion architecture, data quality controls, and CI/CD processes in a production environment.
You will collaborate with architects, BI teams, QA, and business analysts to ensure robust pipelines
#J-*****-Ljbffr
Requirements
- Senior-level experience designing, developing, and optimizing enterprise data pipelines
- Azure Databricks
- Azure Data Lake Storage (ADLS)
- Azure Data Factory (ADF)
- PostgreSQL
- Azure DevOps
- Implementing Medallion architecture
- Building ingestion, transformation, and reporting-ready datasets
- Data quality controls
- CI/CD processes in a production environment
- Collaboration with architects, BI teams, QA, and business analysts