Back to SearchSenior Big Data Engineer with Python
Sorry, this position is no longer available
We are looking for an experienced remote Senior Big Data Engineer with a passion for data to develop, monitor, and operate the curated data pipeline for our project.
The ideal candidate has a strong background in Data Software Engineering and extensive experience with DSE Python AWS Databricks and SQL.
In this position, you will be responsible for redeveloping legacy pipelines into new, advanced, and scalable versions, consulting with data scientists and product managers, building and improving KPIs, and maintaining our cloud-based tech stack.
Responsibilities
- Develop, monitor, and operate the most critical curated data pipeline in the project
- Consult with data scientists and product managers to improve KPIs for business steering
- Redevelop legacy pipelines into advanced, scalable versions that are easy to maintain
- Leverage and improve our cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala
- Build, monitor, and maintain Apache Airflow pipelines
Requirements
- Minimum of 3 years of experience in Data Software Engineering
- Expertise in Apache Spark, Spark streaming, and Databricks
- Fluency in Scala programming language and SQL
- Experienced working with AWS landscape and Github
- Ability to build Apache Airflow pipelines
- B2+ English level
Nice to have
- Knowledge of Presto, Superset, and Starburst
- Experience with Oracle & Exasol
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn