About the role
Applaudo in Peru is seeking a Senior Data Engineer to design, build, and maintain production-grade data pipelines using Azure Databricks and PySpark. You will connect data across enterprise systems, ingest from APIs, databases, and cloud storage, and ensure data quality.
The role emphasizes ownership, documentation, and collaboration with a US-based team, delivering scalable transformations and reliable datasets to empower business insights.
Design, build, and maintain reliable, production-grade data pipelines using Azure Databricks and Python. Develop scalable data transformations using PySpark or Spark SQL to ingest data from multiple systems. Ingest data from APIs, relational databases, cloud storage, and SFTP sources. Integrate data from enterprise platforms into a centralized Data Hub and maintain data warehouses. Write and optimize SQL transformations, ensure data accuracy and processing efficiency. Document pipelines, dependencies, and processes; troubleshoot issues and communicate progress. 5+ years of professional experience in Data Engineering, designing, building, and maintaining production-grade data pipelines. Strong hands-on experience with Azure Databricks. Advanced Python skills for data processing, transformation, and pipeline development. Solid experience with Apache Spark, including PySpark and Spark SQL, for processing data at scale. Advanced SQL proficiency, including complex transformations, data validation, and query performance optimization. Experience integrating data from multiple sources (REST APIs, relational databases, cloud storage, SFTP) and Azure data services. Knowledge of data warehousing and dimensional modeling.