Back to SearchSenior Big Data Engineer with Databricks
Data Software Engineering, Amazon Web Services, Apache Airflow, Apache Spark, Data Warehousing, Databricks, SQL
Sorry, this position is no longer available
We are looking for an experienced remote Senior Big Data Engineer with expertise in Databricks to lead our team of data engineers and take responsibility for designing and implementing data management solutions.
The successful candidate will be familiar with cloud-based data pipeline development and have experience building optimal data infrastructure solutions.
Responsibilities
- Lead a team of data engineers, organize teamwork, and solve technical and non-technical challenges
- Design key software components of the data platform, and build collaborative partnerships with architects and technical leads
- Create and maintain optimal on-prem and cloud-based data pipeline architectures capable of ingesting structured and unstructured data using batch & streaming methods
- Assemble large, complex data sets that meet functional and technical requirements, and develop data flows that can leverage both on-premise and cloud architectures
- Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs
Requirements
- Bachelor's degree in Computer Science, Information Systems, or another applicable field is preferred
- At least 3 years of experience in a Data Engineer role
- Advanced working knowledge of cloud-based data pipeline development, particularly on AWS
- Experience building and optimizing data pipelines for data acquisition and management
- Expertise in Databricks, Python, SQL, and AWS
- B2+ English level
Nice to have
- Understanding of message queuing, stream processing, and Change Data Capture mechanisms
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn