About the role
Applaudo is seeking a Senior Data Engineer in Medellín to design, build, and maintain production-grade data pipelines using Azure Databricks and Python. You will develop scalable PySpark/Spark SQL transformations and integrate data from APIs, databases, cloud storage, and SFTP sources into a centralized Data Hub.
Strong SQL, data warehousing knowledge, and experience with Azure data services are required. You will own data quality validations, troubleshoot issues, and collaborate with a US-based
Design, build, and maintain production data pipelines using Azure Databricks and Python. Develop scalable transformations with PySpark or Spark SQL. Ingest data from APIs, relational databases, cloud storage, and SFTP sources. Create and maintain centralized data hub integrating multiple enterprise platforms. Develop and optimize data warehouse structures and data marts. Write and optimize SQL transformations for accuracy and performance. Implement data quality checks and validate source-target consistency. Investigate data issues and implement corrective actions. Document pipelines, logic, dependencies, and processes. 5+ years in Data Engineering, building production-grade pipelines. Azure Databricks, Python, Spark experience required. Strong SQL and data warehousing/dimensional modeling knowledge. Experience integrating data from APIs, databases, cloud storage, and SFTP. English communication with a US-based team.