Back to SearchLead Data Software Engineer with Databricks, Apache Kafka, Apache Spark, Kubernetes
Remote in Türkiye
Data Software Engineering
Looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.
Find me a job
We are seeking a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as DataFactory, Databricks, EventHub, and Cassandra, to a cloud-agnostic platform capable of on-premises deployment.
Responsibilities
- Implement streaming and batch Spark pipelines running on K8S
- Develop objects of the data generator framework
- Deploy and test solutions on local and development environments
- Enhance and refine the current actively evolving solution
- Investigate and resolve bugs
- Conduct unit testing to ensure code quality
- Participate in refinement, planning, and demo sessions
- Guide and mentor team members while syncing with the team lead
Requirements
- 5+ years of experience with Apache Spark, Databricks, and Kubernetes (K8S)
- Proficiency in Python
- Experience with Confluent/Apache Kafka
- Completed EPAM Data Engineering course or equivalent qualification
- Capability to quickly adapt and dive into the active implementation phase of a project
- Ability to work independently while coordinating with a team lead
- Strong communication skills and a proactive, hands-on, team-player attitude
- English proficiency at B2 level or higher
Nice to have
- Knowledge of Java
- Familiarity with Kafka Connect, KSQL, and Kafka Streams
- Background in Ansible and Argo CD
- Understanding of TDD