In short
Senior Data Engineer who leads development of large-scale data pipelines and architectures, provides technical leadership on data projects, and ensures data quality across environments. Core stack: Python, PySpark, SQL, Bash, and AWS data services (S3, Redshift, Lambda, EMR, Airflow, Glue) in a healthcare insurance domain.
About the role
- Strong experience in Python Scripting and PySpark for data processing
- Proficiency in SQL, dealing with big data processing
- Experience in AWS cloud platform and its data services (S3, Redshift, Lambda, EMR, Airflow, Glue)
- Expertise in BASH/Shell scripting
- Strong understanding of healthcare insurance domain and experience leading data engineering teams
- Experience in Agile environments.
- Excellent problem-solving skills and attention to detail.
- Effective communication and collaboration skills.
Responsibilities:
- Leads development of data pipelines and architectures that handle large-scale data sets.
- Designs, constructs, and tests data architecture aligned with business requirements.
- Provides technical leadership for data projects, ensuring best practices and high-qualitydata solutions.
- Collaborates with product, finance, and other business units to ensure data pipelines meet business requirements.
- Oversees development of data solutions that enable predictive and prescriptive analytics.
- Ensures the technical quality of solutions, managing data as it moves across environments.
Requirements
- Strong experience in Python scripting and PySpark for data processing
- Proficiency in SQL, dealing with big data processing
- Experience with AWS cloud platform and its data services (S3, Redshift, Lambda, EMR, Airflow, Glue)
- Expertise in BASH/Shell scripting
- Strong understanding of healthcare insurance domain
- Experience leading data engineering teams