In short
Contract Data Engineer (outside IR35, ~6 weeks, mostly remote with initial 2-3 on-site days in Exeter) building data matching pipelines on Microsoft Fabric — cleansing and standardising customer data, tuning matching models with PySpark and Splink, and creating master customer records across systems.
About the role
Data Engineer
Contract - Outside IR35 | £400-£450 per day | 6 Weeks Inital Contract Length | Remote (Inital 2-3 days on-site to kick start the project)
Our client is looking for an experienced Data Engineer with strong Microsoft Fabric, PySpark and Splink experience. You'll be responsible for developing data matching pipelines, cleansing and standardising customer data, tuning matching models and building accurate master customer records across multiple systems.
- Experience building and supporting data pipelines in a cloud data platform, ideally Microsoft Fabric.
- Strong Python, including pandas and PySpark.
- Strong SQL, including Spark SQL or T-SQL.
- Experience cleaning and standardising data
- Experience matching, deduplicating or reconciling records across systems, whether rule-based or probabilistic.
- Understanding of how to measure data quality and match accuracy, including false positives and missed matches.
- Good working knowledge of UK GDPR and handling personal data securely.
- Use of version control (Git) and writing readable, documented, tested code.
- Ability to explain technical decisions and trade-offs clearly to non-technical colleagues.
Requirements
- Experience building and supporting data pipelines in a cloud data platform, ideally Microsoft Fabric
- Strong Python, including pandas and PySpark
- Strong SQL, including Spark SQL or T-SQL
- Experience cleaning and standardising data
- Experience matching, deduplicating or reconciling records across systems, rule-based or probabilistic