Senior Data Software Engineer

Data Software Engineering

Location-specific conditions & benefits*

Portugal

We are seeking a highly skilled and motivated Senior Data Software Engineer to join our team, focusing on the development of a next-generation framework for centralized platform infrastructure.

You will drive the implementation of core infrastructure components, manage the integration of parser pipelines, and ensure operational excellence by reinforcing best practices in monitoring, cost optimization, and event-based architectures.

Responsibilities

Be a senior member of a team involved into key engineering activties
Design and implement scalable, centralized platform architecture for data pipelines management
Develop solutions to process tabular and unstructured datasets using frameworks integrated with Unity Catalog tables and volumes
Support event-driven file ingestion from the raw data layer using cloud-native technologies
Ensure robust logging and monitoring by establishing centralized observability practices
Troubleshoot performance bottlenecks and implement improvements for data pipelines
Collaborate with teams to optimize framework performance and implement cost control measures
Participate in the integration of modern infrastructure tools and solutions into the framework
Provide technical mentorship and contribute to coding standards and best practices
Coordinate with stakeholders for seamless onboarding across diverse parser pipelines
Stay updated on emerging technologies to align framework capabilities with future requirements

Requirements

3+ years of experience in data software engineering with a focus on data platform or high-load data pipelines
Skills in PySpark for batch and streaming workloads.
Experience with Azure Databricks, Unity Catalog, Delta Live Tables + Databricks Asset Bundles
Proficiency in Python, experience with using it in OOP fashion
Strong understanding of cloud computing concepts with a preference for Azure skills
Working experience with Terraform for IaC and Azure DevOps for repos, pipelines, and artifacts management (or alternative technologies)

Nice to have

Familiarity with Unity Catalog, knowledge of Fine Grained Authorization capabilities
Experience with Azure Kubernetes Service
Experience with implementing cost control strategies in large-scale cloud systems
Background in using observability tools for centralized logging and monitoring

We offer/Benefits

International projects with top brands
Work with global teams of highly skilled, diverse peers
Healthcare benefits
Employee financial programs
Paid time off and sick leave
Upskilling, reskilling and certification courses
Unlimited access to the LinkedIn Learning library and 22,000+ courses
Global career opportunities
Volunteer and community involvement opportunities
EPAM Employee Groups
Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn