Skip To Main Content
backBack to Search

Senior Data Software Engineer

Hybrid in Portugal
Data Software Engineering
& 6 others

We are seeking a talented and driven Senior Data Software Engineer to join our team.

In this role, you will harness your expertise in cloud-native technologies like Databricks, Azure DevOps, Delta Lake, and Spark Structured Streaming within a Microsoft Azure environment. You will play a pivotal part in designing, implementing, and maintaining scalable, performant, and transparent data-processing pipelines for critical financial data, pushing the boundaries of system scalability and efficiency while collaborating across teams.

If you are eager to take on an exciting challenge in a collaborative and innovative environment, we want to hear from you!

Office presence of about 3 days a week is desirable.

Responsibilities
  • Develop, maintain, and optimize efficient and scalable data pipelines using Python and Apache Spark
  • Build streaming and batch solutions leveraging Delta Lake tables and Spark Structured Streaming
  • Collaborate closely with business partners, IT experts, and architects to gather requirements, design solutions, and refine processes
  • Participate in peer code reviews to ensure code quality and adherence to best practices
  • Utilize relational data models and SQL to manage, query, and process data effectively
  • Implement automated testing strategies for data solutions to ensure high reliability and maintainability
  • Monitor, troubleshoot, and resolve complex data processing issues to support operational stability
  • Document technical designs, processes, and decisions with clarity and precision
  • Proactively identify areas for improvement, recommending and implementing solutions
  • Engage with product owners and team members to prioritize and deliver high-impact solutions
Requirements
  • 3+ years of experience in a software engineering or data engineering role working with large-scale data systems
  • Significant experience in Python, Apache Spark, and Delta Lake
  • Knowledge of relational data models, SQL, and using them for complex data handling
  • Expertise in cloud-native technologies within the Microsoft Azure ecosystem including Databricks and Azure DevOps
  • Familiarity with automated testing practices and the ability to design reliable test strategies
  • Proficiency in analyzing complex problems, breaking them into actionable steps, and delivering elegant solutions
  • Fluency in English and strong communication skills to work effectively within distributed and collaborative teams
Nice to have
  • Understanding of Spark Structured Streaming for real-time and near real-time data processing solutions
  • Showcase of working on financial data or similar high-complexity data domains
  • Competency in identifying and resolving performance bottlenecks in distributed data systems
  • Familiarity with documenting data pipeline architecture and best practice implementation approaches
Benefits
  • International projects with top brands
  • Work with global teams of highly skilled, diverse peers
  • Healthcare benefits
  • Employee financial programs
  • Paid time off and sick leave
  • Upskilling, reskilling and certification courses
  • Unlimited access to the LinkedIn Learning library and 22,000+ courses
  • Global career opportunities
  • Volunteer and community involvement opportunities
  • EPAM Employee Groups
  • Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn