Lead Data Software Engineer (Databricks)

Hybrid in Greece

Data Software Engineering

Facebook LinkedIn Send via email

Looking for something else?

Find a vacancy that works for you. Send us your CV to receive a personalized offer.

Find me a job

We are searching for a Lead Data Software Engineer to join our team and oversee the development of our cutting-edge supply-chain data analytics platform.

This platform delivers a comprehensive view of suppliers, products, material categories, shipments, and compliance activities while employing artificial intelligence to generate insights and recommendations tailored to our clients’ needs.

If this role excites you – apply!

Responsibilities

Design and develop data architectures and ETL pipelines utilizing Databricks and external orchestrators such as Airflow
Collaborate with machine learning teams to incorporate AI-driven insights into the platform
Develop system-level improvements and components for data engineering workflows
Optimize the performance and scalability of data-intensive applications
Adhere to software engineering standards, including containerization, unit testing, linting, and code style reviews
Maintain and improve data workflow, Delta Lake, and Delta Live Tables within Databricks
Provide technical mentorship and leadership to the data engineering team
Engage with clients and stakeholders to understand their requirements and offer customized solutions
Tackle challenges independently and ensure successful project delivery

Requirements

Experience leading data engineering efforts on projects utilizing Databricks
Solid understanding of data architectures and skills in data modeling
Background in designing and building ETL pipelines in Databricks with external orchestrators such as Airflow
Hands-on expertise in Databricks (Delta Lake, workflows, Delta Live Tables, deployment, and versioning)
Proficiency in Python and cloud-native technologies
Competency in Spark/PySpark
Engineering experience with either AWS or Azure
Knowledge of big data and techniques for optimizing data-intensive applications
Strong communication skills with a proactive attitude and client-facing experience
Ability to navigate ambiguity and complete projects independently
Comfort working in a dynamic and transparent startup environment
Fluent English communication skills at a C1 level

Nice to have

Proficiency in configuring and managing CI/CD pipelines in Azure DevOps
Familiarity with Data Observability and Data Quality Monitoring practices
Capability to integrate data quality checks into data pipelines