Senior Data Software Engineer (Databricks)
Data Integration
& 3 others
Argentina
We are seeking an accomplished and motivated Senior Data Software Engineer with expertise in Databricks and data streaming technologies to become a valuable part of our innovative team.
This position requires leveraging skills in big data engineering, cloud platforms, and real-time data streaming to create scalable, efficient, and reliable data systems that support critical business insights.
Responsibilities
- Implement data pipelines in Databricks following medallion architecture principles for optimal data organization
- Optimize batch and streaming pipelines, utilizing Stream Tables, Delta Live Tables, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD)
- Maintain Databricks Asset Bundles (DABs) for efficient deployment, packaging, and version management
- Schedule and manage workflows and job orchestration on Databricks to ensure operational reliability
- Design real-time data platforms with technologies such as Apache Kafka, Confluent, and Redpanda
- Utilize a Schema Registry to enforce data contracts and maintain schema compatibility
- Develop scalable data processing systems using Spark, SQL, and Python
- Apply knowledge of relational and non-relational databases, including MySQL, PostgreSQL, and DynamoDB
- Enhance database query performance for both operational and analytical purposes
- Collaborate with multidisciplinary teams to gather requirements and deliver tailored data solutions
- Adhere to high engineering standards using CI/CD pipelines and Git for version control
Requirements
- Minimum of 3 years of experience in data software engineering
- Hands-on expertise with Databricks, covering Spark, Delta Lake, Unity Catalog, and workflows
- Background in developing ETL/ELT processes, including batch and stream pipelines, CDC, and Slowly Changing Dimensions (SCD)
- Advanced proficiency in Spark programming, SQL query optimization, and Python programming
- Practical knowledge of stream-based architectures using technologies like Kafka, Confluent, or Redpanda
- Strong understanding of cloud platforms such as AWS or GCP for managing data infrastructure
- Competency in working with databases such as MySQL, PostgreSQL, and DynamoDB
- Familiarity with data modeling approaches like star and snowflake schemas for analytical solutions
- Experience with CI/CD pipelines, Git-based version control, and tools such as Terraform for infrastructure automation
- Exceptional ability to solve technical challenges effectively
- Strong communication skills for collaborating across technical and non-technical teams
Nice to have
- Knowledge of data governance practices and regulatory standards such as GDPR, CCPA, or SOC2
- Familiarity with additional big data solutions, including Apache Hadoop or Snowflake
- Relevant qualifications like Databricks Data Engineer Associate or AWS Cloud certifications
We offer/Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn