Back to SearchSenior Big Data Engineer with Databricks
Data Software Engineering, Amazon Web Services, Apache Airflow, Apache Spark, Databricks, Python, Scala
Sorry, this position is no longer available
We are seeking a remote Senior Big Data Engineer with expertise in Databricks to join our innovative team.
The ideal candidate should have extensive experience in Spark/Scala and AWS, as well as a solid foundation in Python.
In this position, you will be responsible for designing, developing, and deploying large-scale data processing systems with Databricks. You will also be responsible for optimizing data pipelines, monitoring performance, and resolving any related issues. The successful candidate should be able to collaborate closely with other teams, evaluate technical trade-offs, and ensure high-quality solutions are delivered.
Responsibilities
- Design, develop and maintain high-performance data processing systems using Databricks
- Optimize data pipelines and monitor their performance regularly
- Identify and troubleshoot issues that occur during data processing
- Collaborate with other teams to integrate data solutions into existing platforms
- Evaluate technical trade-offs and provide recommendations to improve the platform's performance and scalability
Requirements
- Minimum 3 years of relevant experience in Data Software engineering
- Expertise in DSE Python, AWS, and Databricks
- Solid foundation in Spark/Scala and Elastic
- Excellent communication skills and the ability to collaborate with cross-functional teams
- Fluent in English with an Upper-Intermediate level (B2+)
Nice to have
- Experience in Apache Airflow and Python
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn