In short
A data engineer role in Lille designing and industrializing data pipelines on AWS and Databricks using Python, PySpark, and SQL, ensuring data reliability with dbt, orchestrating with Airflow/Astro Scheduler, and handling CI/CD via GitHub in an Agile setting.
About the role
KUBE Partners S.L. - Pixie Services recherche un Data Engineer pour intervenir sur des projets Data Engineering et développer des plateformes Data dans un environnement Cloud AWS et Databricks.
Le consultant concevra et industrialisera des pipelines de données en Python et PySpark, optimisera les requêtes SQL et assurera la fiabilité des données avec Databricks et DBT. L’orchestration se fera via Airflow/Astro Scheduler et le CI/CD via GitHub, dans un cadre Agile.
Requirements
- Data Engineering experience building data platforms in AWS Cloud and Databricks
- Design and industrialize data pipelines in Python and PySpark
- Optimize SQL queries
- Ensure data reliability with Databricks and DBT
- Orchestration with Airflow/Astro Scheduler
- CI/CD with GitHub
- Work in an Agile framework