About the role
We are seeking a skilled Data Engineer with 5+ years of experience in designing, developing and maintaining scalable data pipelines and enterprise data solutions. The ideal candidate should have strong proficiency in Python, SQL, ETL/ELT development, cloud technologies, and data integration, along with an understanding of modern AI technologies and data engineering architecture.
Key Responsibilities
- Design, develop and maintain scalable ETL/ELT pipelines using Python, SQL, Pandas and Pyspark to extract, transform and integrate structured and unstructured data.
- Build and manage AWS-based data pipelines using EC2 and S3, integrating data from SQL Server, Snowflake, FileNet and other enterprise data sources.
- Develop data extraction and processing workflows using Amazon Textract, PostgreSQL, pgvector, vector embeddings, and Retrieval-Augmented Generation (RAG) for AI-powered applications.
- Write and optimize complex SQL queries, data transformation logic, and data models to support efficient data processing, storage and analytics.
- Implement data quality checks, validation rules, reconciliation processes, and automated monitoring to ensure data accuracy, reliability, and consistency.
- Develop and maintain analytical datasets, Power BI dashboards, KPIs and KRIs to support business intelligence and operational reporting.
- Collaborate with cross-functional teams to design scalable data architectures, automate workflows, and integrate data engineering solutions with enterprise applications.
- Apply software engineering best practices, including version control, code optimization, documentation, and workflow orchestration using Git, Apache Airflow, and related tools.
Required Qualifications
Bachelor’s Degree or master’s in computer science engineering, Information Technology, Business Analytics or a related technical field.
- 3+ years of professional experience in Data Engineering, Software Development, or related technical roles.
- Strong proficiency in Python, SQL and ETL/ELT pipeline development.
- Experience working with AWS services, particularly EC2 and S3.
- Hands-on experience with relational databases and cloud data warehouses such as SQL Server, MySQL and PostgreSQL and Snowflake.
- Understanding data modeling, data warehousing, data quality, and pipeline orchestration.