About the role
Assignment Description
We are seeking a skilled Data Architect / Team Leader to design and implement scalable data pipelines using Databricks (Spark + Delta Lake) across both batch and streaming workloads. This role includes integrations with Kafka and medallion architecture patterns (Bronze/Silver/Gold).
The successful candidate will:
Develop and orchestrate metadata-driven ingestion frameworks.
Manage custom business logic transformations through Databricks notebooks and workflow orchestration.
Build and maintain semantic data models to support Power BI analytics.
Ensure reliable, governed datasets for business reporting.
Work with event-driven data architectures.
Utilize cloud orchestration tools such as Azure Data Factory (ADF) for automated data ingestion and processing.
Key Requirements:
Strong experience with Databricks, Spark, and Delta Lake.
Familiarity with Kafka and medallion architecture patterns.
Proficiency in developing metadata-driven ingestion frameworks.
Experience in building semantic data models for analytics.
Knowledge of event-driven data architectures.
Working knowledge of Azure Data Factory (ADF).
Experience in migration to Fabric.
Experience in AI is a strong plus.
Other Details:
This project is a remote opportunity with very occasional trips to the offices in Lisbon, allowing for flexible work arrangements. Ideal for professionals aiming to work on cutting-edge data architecture initiatives.