Lead Data Software Engineer
Hybrid in Portugal
Data Software Engineering
& 6 others
Portugal
We are seeking a Lead Data Software Engineer to join our dynamic team, focused on maintaining and operating current data solutions while expanding our framework to accommodate evolving business needs.
This position provides the chance to contribute to critical projects involving operations, workload migrations, and advancements in the data ecosystem.
Responsibilities
- Take responsibility for access management, onboarding tasks, and pipeline monitoring to maintain operational stability
- Optimize expenses and manage the lifecycle processes for Azure services
- Coordinate data replication efforts across solutions with precision and dependability
- Oversee workload migrations between GTRF 1.0 and GTRF 2.0 systems
- Offer technical insights to enhance framework capabilities in response to business requirements
- Work closely with cross-functional teams to refine solution architecture and scalability
- Execute infrastructure and configuration updates using tools such as Terraform
- Evaluate performance metrics, identify issues, and recommend strategies for system improvements
- Ensure alignment with established best practices in cloud and data engineering
Requirements
- 5+ years of experience in Azure DevOps, Databricks, and PySpark
- 1+ years of relevant leadership experience
- Proficiency in Python for data workflows and processing
- Background in infrastructure automation and deployment with Terraform
- Understanding of pipeline development and monitoring in cloud infrastructures
- Capability to manage operational priorities like access control and cost efficiency
Nice to have
- Familiarity with Databricks SDK for application development
- Practical experience with Databricks API for seamless integration
- Skills in improving collaborative tools within a Databricks-centered environment
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn