About the role
- Strong hands-on experience building and operating AWS-based data platforms and data pipelines.
- Advanced expertise with Apache Spark, PySpark, distributed data processing, and performance optimization.
- Experience with modern data lake technologies, particularly Apache Hudi and Apache Iceberg.
- Strong Python programming and software engineering practices, including testing, code quality, and Git-based development workflows.
- Hands-on experience with Apache Airflow or comparable workflow orchestration platforms.
- Experience with Terraform, CI/CD pipelines, and infrastructure automation.
- Ability to independently translate business requirements into production-ready data solutions.