About the role
Data Platform Engineer
Freelance Contract ( one year followed with extensions)
Location: Brussels
Hybrid model: 2 option: Nearsite only 30 days per year Onsite OR 50% Onsite (different rates)
About the Team
Our data platform team gives domain teams a self-service, governed environment to build and share data products, following the data mesh philosophy. The platform is the backbone of the data-driven solutions we build with our functional experts to make European aviation safer and more efficient. You will work in a Scrum team.
The Role
You build and automate the backbone of the platform. Everything a domain team needs to work independently is provisioned, secured, monitored and maintained through code, and you are the engineer who makes that happen. Networking and security run through all of it: every domain must be isolated, every connection private, and every access controlled. You work closely with the platform architect, who sets the design, and with the domain teams who use what you build.
Our current stack is Azure Databricks with Unity Catalog, automated with Terraform, Databricks Asset Bundles, Python and Azure DevOps, on Azure landing zones with hub-spoke net working and separate DEV, TEST, PREPROD and PROD environments. Platform services includeKey Vault, Azure Policy, Azure Monitor and Log Analytics, with Collibra for the data catalog. We value demonstrated experience over familiarity with the concepts.
What You’ll Do
- Automate the provisioning of environments, workspaces, networking and access for each domain, so onboarding a new team is routine.
- Implement the workspace and catalogue lay out,in line with the architect’s design, that keeps domains isolated and their data governed.
- Implement and maintain the network isolation, private connectivity, identity and access controls that secure the platform.
- Build and run the CI/CD pipelines that deploy platform components and domain workloads.
- Implement cost visibility, so every domain can see and control what its data products cost.
- Implement monitoring and observability for the platform and the data products running on it.
- Build the shared data quality capability that runs alongside domain pipelines and reports centrally.
- Operate the platform in production: upgrades, incidents, performance and security.
- Take part in the Scrum events of the team: refine, plan and deliver backlog items sprint after sprint, and demonstrate the results.
What Success Looks Like in the First Year
- A new domain is onboarded end to end through automation, with no manual steps.
- Cost, health and quality of every data product are visible without manual effort.
- The platform runs predictably, with fewer incidents and known failuremodes handled by automation.
What We’re Looking For
- 5+ years in cloud or platform engineering, including 3+ years runninga data platform in production.
- Strong Azure networking and security: virtual networks, private endpoints, DNS, firewalls, Entra ID, RBAC, Key Vault, Azure Policy.
- Solid AZURE operations experience: resourceorganisation, monitoring, and cost management including tagging,cost allocation and optimisation, with the judgement to choose simple,cost-effective solutions.
- Hands-on Databricks administration: workspace design, Unity Catalog, cluster policies, system tables, cost and performance tuning.
- Infrastructure as code with Terraform, including state management and the Databricks provider.
- CI/CD for platform and data workloads, for example Azure DevOps pipelines and Databricks Asset Bundles.
- Experience implementing cost management, monitoring or data quality capabilities on a platform.
- Understanding of self-service data platforms and data mesh principles.
- Hands-on experience with streaming and messaging components, for example Event Hubs, Kafka or Service Bus.
- Python development, as much of our platform automation is written in Python, and SQL.
- Experience working in a Scrum team.
- Experience with AI-assisted development, including handling the problems it introduces.
Nice to have: MLOps infrastructure (MLflow, model serving) and the infrastructure for the machine learning lifecycle in line with the platform standards, data governance tooling (Collibra, Purview), Kubernetes, Grafana, Nexus, ITIL Foundation.