Back to Search
We are seeking a Lead Data Integration Engineer to guide the strategic design and development of data products and pipelines on a unified Data & AI platform.
You will work extensively with AWS and Databricks technologies to align data solutions with the platform's vision. In this role, you will lead the evaluation of existing data products, design scalable ETL/ELT pipelines, and mentor the data engineering team to enhance their cloud-native data skills. If you have a strong background in data engineering and leadership, we encourage you to apply and contribute to our innovative data platform.
Responsibilities
- Lead evaluation and discovery of data products to ensure compliance with platform standards and scalability
- Design and architect data products and scalable ETL/ELT pipelines using AWS services and Databricks technologies
- Develop and maintain standardized Databricks notebook templates incorporating AWS data processing and orchestration
- Mentor and support data engineers to improve expertise in AWS-native data services and Databricks development
- Monitor pipeline performance and optimize resource usage with AWS monitoring tools such as CloudWatch and CloudTrail
- Troubleshoot and resolve issues within data pipelines proactively
- Participate in cross-functional design sessions providing expertise on Spark, Delta Lake, MLflow, and cloud data ecosystem
- Implement data quality monitoring and pipeline optimization solutions
- Establish and enforce technology standards and reusable components for data engineering teams
Requirements
- Experience of 5 or more years in data engineering
- Leadership experience of at least 1 year in relevant roles
- Proven expertise in designing data pipelines and products using Databricks and AWS native data services including AWS Delta Sharing
- Strong proficiency in Apache Spark, Delta Lake, Python, SQL, and Databricks notebook development
- Deep understanding of AWS cloud architecture, infrastructure as code, automation, security, and monitoring tools
- Experience in setting technology standards and reusable templates for data engineering
- Demonstrated leadership skills in mentoring, collaboration, and promoting cloud-native data engineering practices
- Familiarity with data governance frameworks and use of AWS Lake Formation, Delta Sharing, and IAM in production
- Advanced proficiency in English (B2+/C1)
Nice to have
- Databricks Certified Professional Data Engineer certification
- AWS Certified Solutions Architect or DevOps Engineer certification
Looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.
Find me a job