About the role
UST USource is seeking a highly skilled Lead Data Engineer to join our Data & Analytics team in Malaysia. You will design, develop, and manage end-to-end data solutions on Microsoft Azure, enabling reliable ingestion, transformation, storage, and analytics for business operations and decision making.
The role emphasizes strong hands-on experience with ADF, Synapse, Databricks, and data warehousing, with collaboration across business and technical teams to deliver scalable data platforms.
Design and develop end-to-end data pipelines using Azure Data Factory. Ingest data from REST APIs, SharePoint, Excel, and ADLS Gen2. Build scalable ETL/ELT solutions using ADF Mapping Data Flows and metadata-driven frameworks. Develop and optimize data models, data warehouses, and analytics solutions using Azure Synapse Analytics. Leverage Azure Databricks and Apache Spark for large-scale data processing. Implement data quality, validation, and monitoring processes to ensure reliable datasets. Collaborate with business and technical teams to deliver robust data solutions. Monitor, troubleshoot, and enhance production data pipelines. 8+ years of hands-on experience with Azure Data Factory (ADF), including pipeline orchestration and Mapping Data Flows. Experience with Azure Synapse Analytics, ADLS Gen2, and Azure Databricks. Expertise in REST API integrations, authentication, pagination, and error handling. Strong knowledge of ETL/ELT, data pipeline design, incremental loading, and data validation. Experience in Data Warehousing, Dimensional Modeling, Star Schema, and Snowflake Schema. Good understanding of metadata-driven architecture and scalable cloud data platforms.