Senior Big Data Engineer with Snowflake

Data Software Engineering

Location-specific conditions & benefits*

Colombia

We are seeking a highly skilled Senior Big Data Engineer to join our remote team, working on exciting projects in the field of data engineering and analytics.

In this position, you will be responsible for writing batch processing pipelines using Spark Scala, collaborating with ingestion teams to create new pipelines, troubleshooting and fixing production issues, testing the created code, implementing unit and integration tests, and deploying solutions to production. You will also be responsible for data modeling, and also reading data from Kafka, and processing it in batch mode.

Responsibilities

Write batch processing pipelines, collaborating with ingestion teams to create new pipelines
Troubleshoot and fix production issues, testing the created code, and implementing unit and integration tests
Do data modeling for new data products and read data from Kafka and process it in batch mode
Deploy solutions to production and ensure smooth operation of the pipelines
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
Ensure data quality, integrity, and security throughout the data processing lifecycle
Contribute to the development of best practices and standards for Big Data engineering and analytics

Requirements

A minimum of 3 years of experience in Data Software Engineering, with a focus on Big Data technologies and platforms
Expertise in Snowflake, Amazon Web Services, Python, and SQL
Strong experience with Apache Hadoop, Apache Kafka, Apache Spark, and Apache Impala
Experience in writing batch processing pipelines using Spark Scala, and deploying solutions to production
Strong skills in data modeling for new data products
Good understanding of data ingestion, processing, and storage concepts
Hands-on experience in testing the created code, implementing unit and integration tests, and troubleshooting production issues
Excellent communication skills and strong critical thinking capabilities to effectively convey feedback and insights
Fluent spoken and written English at an Upper-Intermediate level or higher, enabling effective communication

Nice to have

Experience with real-time data processing technologies such as Apache Flink or Apache Storm
Knowledge of containerization technologies such as Docker and Kubernetes
Experience in working with NoSQL databases such as MongoDB or Cassandra
Familiarity with machine learning concepts and frameworks such as TensorFlow or PyTorch
Experience with data visualization tools such as Tableau or Power BI

Benefits

International projects with top brands
Work with global teams of highly skilled, diverse peers
Healthcare benefits
Employee financial programs
Paid time off and sick leave
Upskilling, reskilling and certification courses
Unlimited access to the LinkedIn Learning library and 22,000+ courses
Global career opportunities
Volunteer and community involvement opportunities
EPAM Employee Groups
Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn