About the role
About the Role
As a Senior Data Engineer, you will lead the design, development, and maintenance of data pipelines and infrastructure. You will work with data engineering teams, vendors, and internal stakeholders to ensure data solutions are scalable, efficient, and aligned with business requirements.
You will also contribute to architectural decisions, technical improvements, and engineering best practices.
Key Responsibilities
- Design, develop, and maintain ETL/ELT pipelines that extract, transform, and load data from multiple sources into data warehouses.
- Design and implement data models that support business requirements and efficient querying and analysis.
- Lead the development of scalable and reliable data orchestration pipelines.
- Manage workflow automation, task dependencies, and data transformations using Apache Airflow and DBT.
- Build and maintain real-time streaming pipelines using technologies such as Kafka, Pub/Sub, PySpark, and Dataflows.
- Collaborate with engineering teams, vendors, and stakeholders to continuously improve data products and systems.
- Apply data engineering best practices covering coding standards, DevOps, documentation, and metadata standardization.
- Evaluate architectural and data modeling solutions, including their feasibility, trade-offs, and limitations.
- Support cloud migration initiatives and ensure effective cloud deployment practices.
- Oversee engineering quality, testing, and project delivery timelines.
- Document existing and future solution designs to strengthen technical knowledge across the team.
- Leverage AI tools to improve productivity and focus on higher-impact architectural work.
Must-Have Qualifications
- At least 5 years of experience in a Senior Data Engineer capacity.
- Strong proficiency in Python programming.
- Proficiency in SQL.
- Solid understanding of OLTP concepts, including normalized schema design.
- Solid understanding of OLAP concepts and data modeling.
- Experience designing and building data pipelines, ETL processes, and managing data infrastructure.
- Experience with Apache Airflow and DBT for workflow orchestration and data transformations.
- Experience with real-time data streaming technologies such as Kafka or Pub/Sub, and PySpark or Dataflows.
- Experience with at least one major cloud platform: Google Cloud Platform (GCP), AWS, or Microsoft Azure.
- Experience with cloud deployment and automated testing using CI/CD solutions such as Cloud Build or Concourse.
- Strong technical leadership, analytical thinking, and cross-functional collaboration capabilities.
Good-to-Have Qualifications
- Experience working with audience tracking, consumer behavior, or subscriber datasets.
- Familiarity with Google Analytics, Adobe Analytics, or Snowplow Analytics.
- Experience managing data involving audience engagement and subscriber behavior.
- Proficiency in using AI tools to improve engineering productivity and efficiency.
- Experience with cloud migration strategies and architectural assessments.