Back to SearchLead Software Engineer (Data Engineering)
Data Software Engineering
& 6 others
Choose an option
We are searching for a skilled Lead Software Engineer with expertise in data engineering to strengthen our team.
Your role will involve creating scalable systems and robust data pipelines to manage and process complex datasets, supporting insightful analysis and innovative data-driven applications.
Responsibilities
- Design backend services and data pipelines to handle and process large-scale datasets from diverse sources
- Collaborate with Product Managers and Engineers to create ML/AI-driven systems for actionable insights and workflow automation
- Build APIs and internal tools to facilitate analytics, reporting, and real-time data visualizations
- Define data models, governance policies, and standards to maintain data integrity and accessibility across teams
- Optimize pipelines using techniques like indexing, caching, and asynchronous processing
- Shape architectural decisions on event-driven systems, distributed workflows, and scalable job orchestration
- Enhance CI/CD workflows, monitoring, and automation to ensure reliable and available data infrastructure
- Contribute to discussions, code reviews, and retrospectives to encourage innovation and improvement
Requirements
- 5+ years of experience with Python, Golang, or PHP for backend development
- 5+ years of experience building and designing backend platforms and services
- 3+ years of experience with data engineering tools such as Airflow, dbt, Spark, or Kafka
- Knowledge of relational databases like MySQL, including data modeling, schema design, and performance tuning
- Background in deploying ML/AI-based solutions for extensive data operations
- Expertise in handling distributed systems and utilizing message queues or stream frameworks
- Competency in developing APIs or services for advanced analytics and reporting
- Proficiency in tools like Redis or Memcached and in technologies such as Docker for containerization
- Familiarity with Linux environments and best practices in DevOps
- Understanding of observability platforms like New Relic or Datadog