About the role
Webellian in Warsaw is seeking a Data Engineer to help build a Central Data Platform for a major global insurer. You will design end-to-end data pipelines on Databricks, manage ETL/ELT processes, and collaborate with data scientists and engineers across teams.
You will work with Azure Cloud technologies, GitHub-based CI/CD, and modern data tooling. A strong background in Python, SQL, and PySpark, plus a degree in a technical field, will help you succeed in this role.
Contribute by building a training dataset for document processing and decision support. Design and implement end-to-end data pipelines (ingestion, transformation, storage, and consumption). Prepare and maintain high-quality training datasets for machine learning models. Work with large-scale data on a modern cloud data platform (Databricks). Apply best practices in data engineering, testing, and deployment. Collaborate closely with data scientists, engineers, and business stakeholders. Continuously improve performance, reliability, and automation of data workflows. Strong experience with Databricks (DBX). Advanced knowledge of Python with libraries such as Pydantic (advantage). Solid experience in building and optimizing ETL/ELT pipelines. Very good knowledge of SQL and relational databases. Experience with PySpark. Familiarity with dbt is a plus. Experience with Azure Data Services is an advantage. Knowledge of CI/CD practices and tools (e.g. GitHub). General understanding of infrastructure, orchestration, and IT security principles.