Back to SearchSenior Data Engineer with Java, Apache Spark
Remote in Georgia, & 4 others
Data Software Engineering
Looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.
Find me a job
Choose an option
We are seeking a Senior Data Engineer to join a technology-driven organization operating in the financial data domain, building a production-grade analytics and data ingestion platform processing large-scale financial datasets. This role focuses on designing, building and operating large-scale data ingestion and analytics solutions for batch and real-time processing of financial data, supporting critical downstream analytics and reporting use cases.
Responsibilities
- Design, develop and maintain Apache Spark batch pipelines and Kafka streaming solutions
- Build serverless microservices using AWS Lambda and Azure Functions
- Develop data ingestion connectors for databases, file systems, message queues and financial data feeds
- Write production-grade code in Java, Scala and Python
- Manage infrastructure using Terraform across multiple environments
- Containerize and deploy applications using Docker and Kubernetes (EKS/AKS)
- Implement and maintain CI/CD pipelines in GitLab CI
- Write and maintain unit and integration tests with high code coverage
- Monitor platform health and troubleshoot production issues in distributed systems
Requirements
- 5+ years of experience in data engineering or backend platform development
- Proficiency in Java and/or Scala as primary languages with Python as secondary
- Expertise in Apache Spark including Spark SQL and Spark Streaming
- Skills in Apache Kafka and event-driven architectures
- Competency in SQL and data modeling
- Familiarity with AWS cloud services and serverless computing
Nice to have
- Knowledge of Terraform for Infrastructure as Code
- Skills in Docker and Kubernetes (EKS/AKS)
- Familiarity with CI/CD pipelines such as GitLab CI, Jenkins or equivalent
- Background in cloud data warehouses such as Amazon Redshift
- Understanding of Microsoft Azure services and Azure-native tooling
- Knowledge of Iceberg, Hudi or Delta Lake