Senior Data Platform Engineer (Databricks)

Hybrid in Spain: Málaga, Spain: Madrid

Data Software Engineering

Facebook LinkedIn Send via email

Looking for something else?

Find a vacancy that works for you. Send us your CV to receive a personalized offer.

Find me a job

Location-specific conditions & benefits*

Choose an option

We are looking for a seasoned Senior Data Platform Engineer to join EPAM, a company at the forefront of shaping digital solutions for Fortune 1000 businesses.

As a critical part of our team, the ideal candidate will bring deep expertise in Databricks and experience across multi-cloud environments. This role will focus on building, optimizing, and maintaining innovative, high-performance data platforms that enable powerful data insights and drive meaningful business transformations.

This position offers a hybrid work setup with flexibility to work from Malaga or Madrid, with occasional office visits required.

Responsibilities

Architect and deploy robust data platforms using Databricks, focusing on optimal performance and security
Create solutions that are cloud-agnostic across AWS, Azure, and GCP to ensure system flexibility and resilience
Design and implement comprehensive data pipelines involving data lakes, warehouses, and streaming technologies
Utilize Databricks SQL, Delta Lake, MLflow, and Spark for data interaction and performance enhancements
Collaborate with various teams to implement and maintain workflows based on Databricks best practices
Develop CI/CD pipelines tailored for data platform deployment and testing
Set up and manage frameworks for monitoring, logging, and alerting to ensure infrastructure health
Optimize compute and storage resources to balance cost-efficiency and performance
Troubleshoot Databricks and Spark performance issues
Mentor team members on effective cluster management and resource allocation in Databricks environments
Maintain compliance and security standards throughout platform operations
Drive adoption of advanced Databricks capabilities like Photon and Graviton instances
Regularly refine architectures to align with evolving business and technology requirements

Requirements

Extensive experience in Databricks, Apache Spark, and distributed data processing systems
Strong programming skills in Python, Scala, SQL
Proficiency in data engineering services on AWS (S3, IAM, Lambda), Azure, GCP
Expertise in data architecture and ETL workflows, focusing on data lakes and lakehouses
Hands-on experience with Terraform, CloudFormation, and CI/CD tools
Familiarity with monitoring tools and observability frameworks for large-scale data environments
Solid communication skills in Spanish (at least C1)

Nice to have

Certifications in Databricks, AWS, Azure, GCP
Knowledge of Kubernetes and containerized deployments for data pipelines
Experience with real-time data streaming frameworks and governance tools