Skip To Main Content
epam-logo-black.svgepam-logo-black.svg
backBack to Search

Lead Data Engineer

Remote in India
Data Software Engineering
& 5 others
warning.png
Sorry, this position is no longer available
India

We are looking for a skilled Lead Data Engineer to join our team. If you are proficient in Python and AWS, and you have experience with migrating GCP BigQuery and DataFlow pipelines into Databricks hosted on AWS, we invite you to apply for this leadership role in driving data engineering excellence.

Responsibilities
  • Lead and oversee the migration of GCP BigQuery and DataFlow pipelines to Databricks hosted on AWS, ensuring a smooth and efficient transition
  • Collaborate with cross-functional teams to design, develop, and optimize data pipelines using Python, Spark, and Databricks, catering to business requirements
  • Provide mentorship and guidance to the data engineering team, fostering a collaborative and growth-oriented environment
  • Architect and implement data models and transformations to facilitate data analysis and reporting
  • Conduct code reviews and quality assurance processes to maintain the reliability and accuracy of data pipelines
  • Troubleshoot and resolve complex data engineering issues, ensuring consistent data flow and integrity
Requirements
  • A minimum of 5 years of relevant experience in Data Engineering, showcasing your expertise in developing and optimizing data solutions
  • At least 1 year of direct leadership and team management experience, highlighting your ability to inspire and guide a technical team
  • Proficiency in AWS, utilizing its resources to build and manage efficient data solutions
  • Proficiency in Python, using it to construct robust and scalable data pipelines
  • Experience with Apache Spark, enabling you to work with large-scale data processing and analysis
  • Expertise in Databricks, especially in the context of migrating pipelines to AWS-hosted platforms
  • Strong command of SQL, allowing you to perform data transformations and analysis effectively
  • Fluent English communication skills at a B2+ level, facilitating effective collaboration with a diverse team
Nice to have
  • amiliarity with Google Cloud Platform (GCP), particularly experience with BigQuery and DataFlow
  • Knowledge of GCP BigQuery and DataFlow, aiding in the migration process and fostering cross-platform collaboration
Benefits
  • International projects with top brands
  • Work with global teams of highly skilled, diverse peers
  • Healthcare benefits
  • Employee financial programs
  • Paid time off and sick leave
  • Upskilling, reskilling and certification courses
  • Unlimited access to the LinkedIn Learning library and 22,000+ courses
  • Global career opportunities
  • Volunteer and community involvement opportunities
  • EPAM Employee Groups
  • Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn

These jobs are for you