Back to SearchRefer a Friend
Know someone who would be a great fit for this opportunity?
We are seeking a visionary and experienced Lead Data Engineer to join our Data Engineering team.
In this role, you will lead a Platform Team, driving the design, development, and scaling of data pipelines in Databricks with PySpark on Microsoft Azure for our AI Factory team. This is a unique opportunity to shape the future of our data platforms, influence technical strategy, and mentor a talented team, all while working at the forefront of big data and cloud engineering.
Responsibilities
- Lead the design and architecture of cloud-native analytical solutions using Big Data and NoSQL technologies
- Oversee the development and optimization of scalable data pipelines in Databricks with PySpark on Azure
- Own the strategy for building and maintaining data lakes and warehouses, ensuring reliability, performance, and scalability
- Define and implement ETL/ELT workflows and best practices for data collection, cleaning, and structuring
- Establish and enforce data quality, lineage, and monitoring frameworks across the team
- Collaborate closely with ML, analytics, and business teams to deliver production-ready datasets and solutions
- Conduct and lead code reviews, setting technical standards and fostering best practices
- Mentor and coach data engineers, cultivating a high-performance and collaborative team culture
- Champion CI/CD methodologies in data engineering workflows using tools like Jenkins or GitLab CI/CD
- Drive requirements gathering and solution alignment with architects, technical leads, and cross-functional teams
- Engage with stakeholders at all levels to understand business processes, model input data, and ensure deliverable alignment with strategic goals
Requirements
- 6+ years of experience in Data Engineering or a related field, with at least 2 years in a technical leadership role
- Deep proficiency in Python and PySpark
- Extensive hands-on experience with Databricks and Microsoft Azure cloud services
- Strong background in software version control tools (e.g., GitHub, Git)
- Proven track record with CI/CD frameworks such as Jenkins, Concourse, or GitLab CI/CD
- Demonstrated expertise in architecting and building scalable, robust, and highly available data solutions
- Excellent problem-solving, analytical, and stakeholder management skills
- Experience in mentoring and leading engineering teams
Nice to have
- Experience with additional programming languages such as Java, SQL, or Scala
- Knowledge of SAP BTP or similar enterprise data platforms
- Familiarity with agile development methodologies
- Experience in strategic planning and technical roadmap development
Looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.
Find me a job
Refer a Friend
Know someone who would be a great fit for this opportunity?