About the role
Senior Data Engineer
A little about this gig
Agility Partners is seeking a qualified Senior Data Engineer to join one of our clients in the financial services industry, supporting Wealth & Asset Management initiatives. This is an exciting opportunity to design and build scalable, secure data solutions that power enterprise applications, analytics, and AI-enabled capabilities. The ideal candidate will bring deep data engineering expertise and hands-on experience with AWS, Python, SQL, and modern data-processing technologies.
- Design, develop, and maintain scalable ETL/ELT pipelines and data-processing workflows to ingest, transform, and integrate structured, semi-structured, and unstructured data from multiple sources.
- Build reusable data-processing components using Python and SQL while designing and supporting data models across PostgreSQL, Amazon RDS, DynamoDB, and Amazon S3.
- Develop data solutions supporting document ingestion, extraction, application integration, analytics, and AI-enabled use cases, including document-processing services like Amazon Textract and vector-based retrieval using PostgreSQL/pgvector.
- Implement data quality, validation, reconciliation, metadata, lineage, governance, security, and monitoring controls while optimizing pipelines and databases for performance, reliability, scalability, and cost.
The ideal candidate
- 8+ years of experience in data engineering, data integration, or a related technical discipline.
- Strong hands-on experience with Python and SQL for data processing, transformation, and pipeline development.
- Experience designing and supporting ETL/ELT pipelines and scalable data-processing workflows.
- Proficiency with PostgreSQL, Amazon RDS, DynamoDB, and Amazon S3 for data storage and management.
- Experience with data modeling, APIs, JSON, Parquet, and pgvector, including structured and unstructured data environments.
- Strong understanding of data quality, validation, metadata, governance, security, encryption, and operational monitoring.
- Experience working in large enterprise or regulated environments, with exposure to document-processing technologies and AI-enabled data solutions.
Reasons to love it
- **Modern Data Engineering:** Work with a variety of AWS services and technologies while building scalable enterprise data solutions.
- **AI Innovation:** Contribute to emerging AI-enabled applications, document processing, and vector-based retrieval capabilities.
- **High-Impact Projects:** Support critical Wealth & Asset Management applications and data initiatives.
- **Technical Collaboration:** Partner with application, AI, DevOps, architecture, cybersecurity, and business teams to deliver innovative data solutions.