About the role
Innovation Hub | Business Intelligence | Senior Data Engineer
Your new Company
Join an international organization building a regional AI Innovation Capability Hub from Lisbon. You will become part of an environment focused on turning Generative AI opportunities into production-ready solutions that can ultimately scale across a wider international business.
Innovationhublx26
Your new role
The Senior Data Engineer will be responsible for designing, developing, and optimizing scalable Data pipelines and architectures within a modern Data platform leveraging Azure and Databricks, ensuring that Data is accessible, reliable, and actionable.
This person will combine strong expertise in Data Engineering, Azure Cloud Services, Databricks, and distributed Data processing with Data Quality, Governance, Security, and DevOps practices, enabling the delivery of scalable, secure, and high-quality Data solutions.
The Senior Data Engineer will collaborate closely with Data Scientists, BI Developers, Analysts, and Business Stakeholders to translate Business requirements into technical solutions and support enterprise-wide analytics and Data initiatives.
The Senior Data Engineer will:
- Design, develop, and optimize scalable Data pipelines, Data lakes, and Delta tables (using Azure Data Factory, Databricks, Apache Spark, Delta Lake, and Azure Data Lake Storage Gen2), ensuring performance, scalability, reliability, and cost efficiency;
- Implement Data Quality, Governance, and Security best practices across Data pipelines and architectures;
- Monitor and troubleshoot Data pipelines and production jobs (using Azure Monitor and Databricks job runs);
- Collaborate with Data Scientists, BI Developers, Analysts, and Business Stakeholders to translate Business requirements into Data Solutions and enable self-service analytics;
- Integrate Data Solutions with Power BI, Power Apps, Microsoft 365 tools, and other enterprise platforms;
- Lead technical workstreams and contribute to reusable pipeline components, development standards, architecture best practices, and peer code reviews;
- Provide mentorship and technical guidance to junior Data Engineers and Analysts, promoting best practices across coding, architecture, and DevOps.
A typical day will include designing and optimizing Data pipelines, collaborating with cross-functional teams, monitoring production environments, troubleshooting issues, and continuously improving Data solutions.
What you will need to succeed
As a Senior Data Engineer, you will need:
- Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field;
- + 5 years of professional experience in Data Engineering, or related roles;
- Azure Certifications (such as Azure Data Engineer Associate, or related certifications – would be a plus);
- Proven experience designing, developing, and optimizing scalable Data pipelines and distributed Data Solutions (using Databricks, Apache Spark (PySpark or Scala), and Delta Lake);
- Experience working with Azure Cloud Services (including Azure Data Factory and Azure Data Lake Storage Gen2);
- Experience programming with SQL and Python, and working with distributed Data processing;
- Experience with CI/CD, Infrastructure as Code, and DevOps practices (using tools such as Terraform, GitHub Actions, or Azure DevOps);
- Experience implementing Data Quality, Governance, Security, Monitoring, and reliability practices across Data pipelines;
- Experience collaborating with technical and non-technical Stakeholders in Global environments, as well as leading technical workstreams and mentoring Junior team members;
- Knowledge of Data Modeling techniques and modern Data engineering practices;
- Familiarity with Data Governance frameworks and tools (such as Azure Purview, Unity Catalog, Data lineage, Data cataloging, and metaData management), as well as real-time Data streaming technologies (such as Kafka or Azure Event Hubs);
- Familiarity with Machine Learning workflows, MLOps on Databricks, Data privacy regulations (such as GDPR or HIPAA), and Agile and Waterfall methodologies (would be a plus);
- Excellent communication skills – Fluency in English.
What the Company can offer you
Have the opportunity to join a cross-functional team in an international company with a multicultural working environment!
Next Steps
If you are interested in this opportunity, please send us your updated CV. If this role is not for you, feel free to share it with your network.
Senior Data Engineer (m/f/d) Hays Working for your tomorrow