About the role
AME Research in Jakarta is seeking a Data Engineer to design, implement, and maintain reliable data pipelines that collect and process data from various public APIs into a centralised blob storage and database environment.
You will ensure data is ingested efficiently and transformed cleanly for downstream analysis and reporting, collaborating with data scientists and developers in an in-office setting.
Design and implement ETL/ELT pipelines to ingest data from multiple public APIs into blob storage and relational databases. Build and manage scheduled workflows using Apache Airflow (or similar). Handle complex API integrations, authentication flows (OAuth2, API keys), pagination, and rate limiting. Structure data in blob storage (e.g., Azure Blob) for efficient downstream access. Transform and load data into structured SQL-based databases for analytics and access by Web applications. Implement monitoring, logging, and validation to ensure data integrity and pipeline reliability. Work closely with data scientists and software developers to ensure data availability and usability. Maintain clear documentation for data sources, transformations, and orchestration logic. Bachelor's degree in a related field or equivalent experience. 3+ years of experience building data pipelines in a production environment. Strong understanding of ETL/ELT architecture, API data ingestion, and data modelling principles. Experience with Azure and cloud-based data ecosystems. Modern office near public transport Competitive salary