Skip To Main Content
epam-logo-black.svgepam-logo-black.svg
backBack to Search

Lead Data Software Engineer

Colombia, Mexico
Data Software Engineering, Amazon Web Services, CI/CD, Python, SQL, Amazon DynamoDB, Apache Hive, Apache Spark, Databricks

We are seeking an experienced and skilled Lead Data Software Engineer to spearhead the development of robust data solutions, drive stakeholder engagement, and lead a high-performing team in creating scalable applications using cutting-edge technologies.

As a pivotal member, you will combine strong technical expertise with leadership capabilities to deliver impactful solutions in a dynamic US timezone environment.

Responsibilities
  • Design and develop scalable data pipelines using Python and SQL
  • Lead by example in application development using microservices architecture, CI/CD, and unit testing frameworks
  • Implement and maintain cloud-based solutions leveraging AWS components like EC2, S3, Lambda, and RDS
  • Manage and optimize data processing workflows with tools such as Hive and Spark
  • Oversee system design, ensuring compliance with data structures, algorithms, and best practices
  • Collaborate with stakeholders to gather requirements and translate them into technical solutions
  • Explore database solutions such as DynamoDB and Elasticsearch to enhance performance
  • Guide the team in building test-driven development strategies, ensuring high-quality code integrity
  • Support data orchestration efforts using tools like Airflow and data warehousing techniques
  • Drive the creation, automation, and troubleshooting of CI/CD pipelines for seamless delivery
  • Mentor and work with teams of 5-10 developers, encouraging professional growth
  • Implement documentation standards and facilitate well-organized technical writing across the team
Requirements
  • 5+ years of working experience in software engineering or data engineering roles
  • Must have proficiency in Python and SQL for application and data development
  • Background in cloud technologies including AWS (EC2, S3, Lambda, DynamoDB, SQS)
  • Expertise in creating unit testing cases using mock frameworks, ensuring robust CI/CD processes
  • Showcase of knowledge in data processing frameworks such as Spark and Hive
  • Competency in data structures, algorithms, and system design
  • Familiarity with Flask, microservices, and pipeline automation processes
  • Understanding of data modeling principles and orchestration tools such as Airflow
  • Flexibility to engage with stakeholders and manage communications effectively
  • Qualifications in test-driven development approaches and constructing scalable data pipelines
Nice to have
  • Familiarity with Elasticsearch for enhancing data retrieval and management
  • Background in DynamoDB database usage within application designs
  • Experience with advanced data modeling and schema optimization techniques
Benefits
  • International projects with top brands
  • Work with global teams of highly skilled, diverse peers
  • Healthcare benefits
  • Employee financial programs
  • Paid time off and sick leave
  • Upskilling, reskilling and certification courses
  • Unlimited access to the LinkedIn Learning library and 22,000+ courses
  • Global career opportunities
  • Volunteer and community involvement opportunities
  • EPAM Employee Groups
  • Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn