In short
PBT Group is hiring a Senior Data Engineer in Johannesburg to design, build and optimise scalable data pipelines and lakehouse platforms on AWS and Databricks (Spark, Delta Lake, Redshift), supporting analytics and reporting; short-term insurance domain experience is a strong advantage.
About the role
Senior AWS Data Engineer job vacancy in Johannesburg.
PBT Group is seeking an experienced Senior Data Engineer with strong AWS and Databricks expertise to join a data-focused team.
The successful candidate will design, build and optimise scalable data pipelines and platforms that support analytics, reporting and advanced data use cases.
Short-term insurance domain experience is a strong advantage.
Key Responsibilities:
- Design, develop and maintain robust, scalable data pipelines using AWS and Databricks.
- Build and optimise ETL/ELT processes for structured and unstructured data.
- Implement and manage data lakehouse architectures (including Medallion or similar layered approaches).
- Ensure data quality, reliability, performance and governance across pipelines.
- Collaborate with data scientists, analysts, architects and business stakeholders.
- Contribute to technical design, documentation and best practices.
- Support production deployments, monitoring and troubleshooting of data solutions.
- Mentor junior engineers and contribute to continuous improvement of the data platform.
Minimum Requirements:
- Solid experience as a Senior Data Engineer.
- Strong hands-on experience with AWS (S3, Glue, Lambda, EMR, Redshift, IAM, or equivalent services).
- Strong hands-on experience with Databricks (including Spark, Delta Lake, notebooks, jobs and Unity Catalog).
- Proficiency in Python and SQL.
- Experience with data modelling, data quality frameworks and pipeline orchestration.
- Familiarity with CI/CD, Git and DevOps practices for data solutions.
- Ability to work in Agile environments and collaborate effectively with cross-functional teams.
- Strong problem-solving and communication skills.
- Advantageous.
- Short-term insurance or financial services domain experience.
- Experience with other cloud platforms or modern data tools (e.g. dbt, Airflow, Kafka).
- Knowledge of data governance, security and cost optimisation on AWS.
- Databricks certifications.
Requirements
- Solid experience as a Senior Data Engineer