About the role
Intuition IT Solutions is seeking a senior Data Engineer to lead the design and delivery of Databricks-based ETL pipelines for manufacturing data. You will profile raw data, implement data quality checks, and build bronze/silver datasets aligned to enterprise standards.
Your role includes master data governance and KPI modeling to support insights and reporting. The position requires strong Python/SQL skills, hands-on Databricks experience, and the ability to mentor teams within an Agile/DevOps
Analyse and profile raw manufacturing and operational data to identify data quality issues and onboarding needs. Design, build, test, and maintain scalable Python-based Databricks ETL notebooks and pipelines. Create reusable bronze and silver datasets conforming to enterprise data architecture. Develop a master data framework aligning products, materials, equipment, and processes. Build KPI calculation frameworks for yield, cycle time, deviations, and process performance. Develop aggregated performance models for trend analysis and reporting. Collaborate with cross-functional teams to validate data and metrics. Implement automated data-quality checks, monitoring, and exception handling. Document mappings, transformation logic, data lineage, and deployment procedures. Work within agile and DevOps practices including CI/CD and production support. Provide technical leadership and mentoring to the delivery team. Bachelor’s degree in Computer Science or related field; equivalent experience may be considered. 7+ years of data engineering experience with data onboarding and modelling deliveries. Advanced hands-on Python and SQL skills, with strong software-engineering practices for modular, testable data solutions. Strong practical experience with Databricks, Apache Spark, notebook-based development, Delta Lake or equivalent lakehouse patterns, and performance optimisation. Demonstrated experience designing scalable ETL/ELT pipelines and layered bronze and silver data products using reusable engineering patterns. Experience designing master data, dimensional, asset-centric, aggregation, and time-series models for analytics and reporting. Strong understanding of data quality, metadata, lineage, reconciliation, observability, access controls, and production support. Experience working with manufacturing and operational data, including batch, product, material, equipment, process, quality, and performance data. Experience with manufacturing execution systems (MES), SAP, historians, laboratory or quality systems, and industrial analytics environments is highly desirable. Knowledge of manufacturing KPIs and the underlying business rules for yield, cycle time, deviations, process performance, and trend-based analysis is highly desirable. Experience with source control, CI/CD, automated testing, agile delivery, and regulated or validated data environments is preferred. Strong analytical, problem-solving, stakeholder-management, and written and verbal communication skills, with the ability to translate operational requirements into robust technical solutions.