Skip To Main Content
backBack to Search

Senior Data Software Engineer (Azure, Spark)

Hybrid in Poland: Gdańsk
Data Software Engineering& 6 others
hot
Looking for something else?

Find a vacancy that works for you. Send us your CV to receive a personalized offer.

Find me a job

We are looking for a Senior Data Software Engineer to contribute to the design and development of a modern enterprise data platform focused on large-scale financial data processing. The platform is being built to deliver new capabilities rather than maintain legacy systems, with a strong emphasis on data quality, governance, and scalable processing across cloud-native technologies. The role involves working on data ingestion, transformation, modeling, and optimization within a medallion architecture, supporting both structured and complex datasets, including private markets data.

Responsibilities
  • Lead the design, development, and optimization of scalable data engineering solutions on Azure, using Spark-based processing (PySpark, SparkSQL, Scala, or equivalent)
  • Own end-to-end data transformation pipelines, including ingestion, transformation, storage, and analytics
  • Work with Azure-native data services such as Data Factory, Databricks, and Synapse
  • Support high-performance data access patterns using Cosmos DB (NoSQL API) where applicable
  • Collaborate with data scientists, AI engineers, and product stakeholders to enable data-driven analytics and insights
  • Mentor and guide junior engineers, setting coding standards and best practices
  • Ensure data quality, security, governance, and performance across platforms
  • Contribute to technical decision-making and solution architecture discussions
  • Build and maintain data pipelines within a medallion architecture, supporting structured and complex datasets
  • Apply CI/CD practices to automate validation and deployment of data pipelines
Requirements
  • 5+ years of experience in data engineering, with a focus on large-scale data processing
  • Proficiency in Microsoft Fabric, Apache Spark, and Python
  • Skills in Pandas, Delta Lake, and Parquet for data processing, storage, and optimization
  • Knowledge of Cosmos DB for bitemporal data storage and retrieval
  • Familiarity with AWS Glue for supporting ETL workflows
  • Expertise in Spark Streaming for near real-time processing with watermarking support
  • Competency in CI/CD tools for building automated validation pipelines
  • Background in mentoring engineers and establishing coding standards and best practices
  • English proficiency at B2 level or higher
Nice to have
  • Hands-on experience with Microsoft Fabric and OneLake
  • Expertise in Cosmos DB (NoSQL API) specifically
  • Familiarity with financial instruments and financial services data
  • Familiarity with AI-assisted development tools such as GitHub Copilot and awareness of industry-standard LLMs
  • Understanding of Data Science fundamentals and collaboration experience with DS teams