About the role
Comviva is seeking a Data Engineer to build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data, and to develop structured datasets for analytics, reporting, and downstream applications.
You will write and optimize SQL for large-scale datasets, implement data transformations with Python or Groovy, and work with Spark, Airflow, NiFi to build scalable workflows while collaborating with stakeholders.
Build and maintain ETL/ELT pipelines for ingesting, transforming, and loading data. Develop and maintain structured datasets for analytics, reporting, and downstream applications. Write and optimize SQL queries to process large-scale datasets efficiently. Implement data transformation logic using Python or Groovy. Work with Spark, Airflow, NiFi, and other platform components to build scalable data workflows. Integrate data from multiple internal and external sources. Perform data validation and identify data quality issues. Troubleshoot pipeline failures and optimize performance. Participate in designing scalable data models and warehouse structures. Collaborate with senior engineers, product teams, and business stakeholders to deliver data solutions. Contribute to automation, monitoring, and operational excellence of the data platform. Follow engineering best practices for testing, documentation, and code quality. 2–4 years of experience in Data Engineering, Backend Engineering, or a related role. Strong proficiency in Python, Groovy, or similar scripting languages. Good understanding of ETL/ELT pipeline development and data transformation techniques. Hands‑on experience with SQL and relational databases. Experience with OLAP databases such as ClickHouse, Amazon Redshift, or Snowflake is preferred. Experience working with Apache Spark, Apache Airflow, or similar distributed data processing and orchestration frameworks. Familiarity with data ingestion tools such as Apache NiFi or equivalent. Experience working with Amazon S3, MinIO, or other object storage systems. Good understanding of data modeling principles for analytics and reporting. Strong analytical and problem-solving skills. Good communication and collaboration skills.