About the role
In this role you will :
- Own reliability and performance of operational pipelines across our product catalog infrastructure.
- Identify and reduce technical debt, replacing reactive patches with designed, testable logic.
- Build and maintain low latency data APIs that serve downstream operational and analytics consumers.
- Implement CDC patterns to keep catalog data synchronized across systems with minimal lag.
- Build monitoring, observability, and automated testing so failures surface before stakeholders report them.
- Design and implement unit standardization and master data logic at catalog scale.
- Translate business requirements from non-technical stakeholders into durable pipeline logic.
- Own code versioning, deployment, and incident response for your layer.
- Leverage your expertise in the tech stack : BigQuery, Databricks, Spark/PySpark, AWS/GCP.
Required :
- 3+ years as a Data Engineer, including solo or primary ownership of production pipelines
- Strong Python - data engineering, transformation logic, testing discipline
- Strong SQL with ability to write correct queries, identify and refactor anti-patterns
- Databricks, Delta Lake, Airflow for production orchestration
- Experience with CDC patterns for real-time or near-real-time data synchronization
- Experience building low latency APIs serving operational or analytical consumers
- Test-driven development discipline - unit tests, integration tests, regression coverage as standard practice, not afterthought
- Operates independently under ambiguity; designs systems to be maintained, not just to run
Preferred :
- Kafka or equivalent event streaming platform experience
- Experience with entity matching, deduplication, or master data management
- Exposure to ML pipeline support in production
- Familiarity with NLP techniques for entity resolution or text normalization (tokenization, similarity matching, named entity recognition)