Back to SearchSenior Data Software Engineer/ AI Agents, Azure, Spark
Find a vacancy that works for you. Send us your CV to receive a personalized offer.
Find me a job
We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem, primarily focused on building and maintaining data transformation pipelines.
The candidate should be comfortable with any Spark-adjacent technology (PySpark, Spark SQL, Scala Spark, Databricks, Synapse, etc.) rather than being locked into one specific flavor. Experience with Microsoft Fabric is a strong plus but not a requirement.
The ideal candidate combines great technical skills with leadership capability, contributing to architecture, design, development, and mentoring of engineering teams — preferably in complex enterprise or financial services environments.
- Lead the design, development, and optimization of scalable data engineering solutions on Azure, using Spark-based processing (PySpark, Spark SQL, Scala, or equivalent)
- Own end-to-end data transformation pipelines, including ingestion, transformation, storage, and analytics
- Work with Azure-native data services such as Data Factory, Databricks, and Synapse
- Support high-performance data access patterns using Cosmos DB (NoSQL API) where applicable
- Collaborate with data scientists, AI engineers, and product stakeholders to enable data-driven analytics and insights
- Mentor and guide junior engineers, setting coding standards and best practices
- Ensure data quality, security, governance, and performance across platforms
- Contribute to technical decision-making and solution architecture discussions
- 3+ years of experience in data engineering roles, preferably within complex enterprise or financial services environments
- Expertise in Azure cloud data services, including Data Factory, Databricks, and Synapse
- Proficiency in the Spark ecosystem, including PySpark, Spark SQL, and Scala Spark
- Background in designing and maintaining end-to-end data transformation pipelines covering ingestion, transformation, storage, and analytics
- Skills in ensuring data quality, security, and governance across large-scale platforms
- Capability to contribute to solution architecture and technical decision-making
- Competency in mentoring engineering teams and setting coding standards and best practices
- Understanding of high-performance data access patterns using Cosmos DB (NoSQL API)
- English proficiency at an Upper-Intermediate level (B2) or higher
- Hands-on experience with Microsoft Fabric and OneLake (Delta / OpenLake)
- Familiarity with financial instruments and financial services data
- Exposure to AI-assisted development tools such as GitHub Copilot and awareness of industry-standard LLMs
- Knowledge of Data Science fundamentals and collaboration experience with DS teams