In short
Data Engineer in Melbourne, Australia, building, enhancing, and maintaining data pipeline frameworks supporting batch, streaming, and near-real-time processing. Daily work centers on ETL/ELT development, SQL-based transformations, and production-grade pipelines using Kafka, PostgreSQL, and AWS.
About the role
Detailed Job Description
Good communication skills with knowledge on Kafka, API implementation, PostgreSQL, AWS
•Strong understanding of SDLC process.
•Should have Strong understanding and trouble shooting skills
Core requirements:
· Must have strong experience in building, enhancing, and maintaining data pipeline frameworks
supporting batch, streaming, and near-real-time processing.
· Must be able to design reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls. · Must demonstrate proficiency in ETL/ELT development, SQL-based transformations, and distributed data processing. Must implement production-grade pipelines with appropriate error handling, logging, monitoring, and performance optimization.
Requirements
- Good communication skills
- Knowledge of Kafka, API implementation, PostgreSQL, AWS
- Strong understanding of SDLC process
- Strong understanding and troubleshooting skills
- Strong experience building, enhancing, and maintaining data pipeline frameworks supporting batch, streaming, and near-real-time processing
- Ability to design reusable data engineering frameworks, accelerators, and templates for ingestion, transformation, orchestration, and data quality controls
- Proficiency in ETL/ELT development, SQL-based transformations, and distributed data processing