Data Engineer in Spain

Viewing 1-10 out of 60 jobs found

Sort By

relevant

hot

Senior Data Engineer

Office in Spain: Málaga

Data Software Engineering

Do you have a software engineering background and strong knowledge and experience in Data? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Senior Data Engineer. EPAM is shaping the digital future for Fortune 1000 companies, building complex solutions using modern technologies. We are looking for a Senior Data Engineer with an open-minded personality who can join our friendly environment and become a core contributor to our team of experts. The candidate needs to have experience in Databricks, Python and SQL to match this position. We are looking for candidates from Malaga or ready to relocate. Successful candidate should be ready to work from office 4 days/week.

responsibilities

Design, develop, monitor, and operate data pipelines
Integrate high-quality datasets for analytical use-cases
Enable other data teams to follow the client standards
Testing (preparation and execution)
Maintenance of existing data pipelines incl. alerting, bug fixing, etc.

requirements

Wide experience in Azure Basis, using Databricks
Databricks delta live / pipeline build experience
Strong skills in Python Core, PySpark and SQL
RESTful services concepts
Understanding of CI/CD
Strong communication skills
Code management tools and experience of working from a checked-in code base

nice to have

TDD concepts awareness

Lead Data Software Engineer (Databricks)

Remote in Spain

Data Software Engineering

We are looking for a Lead Data Software Engineer with a strong background in Databricks and an open-minded personality. This role requires someone ready to join a friendly environment and become a core contributor to our team of experts. Do you have background and wide experience in Data engineering and strong knowledge in Databricks? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Lead Data Software Engineer.

responsibilities

Lead a small team of engineers in designing, developing, and maintaining scalable data pipelines and architectures
Develop and optimize data models and ETL processes using Databricks and other technologies
Implement data quality checks and monitoring to ensure high data integrity
Stay updated with emerging trends in data engineering and recommend new tools when beneficial
Troubleshoot and resolve data-related issues promptly
Participate in code reviews to maintain high standards of code quality

requirements

Hands-on experience with Databricks, including Delta Lake, workflows, Delta Live Tables, deployment, and versioning
Ability to lead a development team of 2-5 engineers
Solid understanding of data architectures and data modelling; experienced in designing and building ETL pipelines with Databricks
Expertise in Spark using (either Scala or PySpark)
Knowledge of SDLC and Agile frameworks
Proficiency in cloud-native technologies and software engineering best practices, including containers, unit tests, linting, and code style checks
Engineering experience with AWS, Azure, or GCP
Experience with big data and performance optimization of data-intensive applications
Proactivity and client-facing experience
Ability to deal with ambiguity and work independently
Desire to work in a transparent and fast-moving startup environment
Fluent in English communication skills at a B2+ level

Lead Data Software Engineer (Databricks)

Office in Spain: Málaga

& 1 others

Data Software Engineering

We are looking for a Lead Data Software Engineer with experience and knowledge in Databricks and open-minded personality, who can join our friendly environment and become a core contributor to our team of experts. Do you have background and wide experience in Data engineering and strong knowledge in Databricks? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Lead Data Software Engineer.

responsibilities

Lead a small team of engineers in creating and maintaining scalable data pipelines and architectures
Develop and optimize data models and ETL processes utilizing Databricks alongside other technologies
Implement data quality checks and monitoring frameworks to ensure high data integrity
Stay abreast with emerging data engineering trends and technologies, advocating for the adoption of new tools when beneficial
Troubleshoot and resolve data-related issues efficiently
Conduct code reviews and uphold high standards of code quality
Foster a collaborative and inclusive environment for the team to grow their skills and expertise
Interact with clients, managing expectations and delivering high-quality solutions that meet their needs
Drive innovation by proposing new data strategies and enhancements
Work collaboratively within an Agile framework to meet project timelines and deliverables

requirements

Hands-on experience with Databricks, including Delta Lake, workflows, Delta Live Tables, deployment, and versioning
Capability to lead a development team of 2-5 engineers
Solid understanding of data architectures and proficient in data modeling, building, and optimizing ETL pipelines with Databricks using external orchestrators like Airflow
Expertise in Spark, using either Scala or PySpark
Knowledge of the software development lifecycle and Agile frameworks
Proficiency in cloud-native technologies and familiarity with engineering best practices including containers, unit tests, linting, and code style checks
Engineering experience in cloud environments with AWS, Azure, or GCP
Experience focused on big data and performance optimization of data-intensive applications
Proactive approach with the ability to operate independently
Excellent client-facing skills and ability to handle ambiguous situations effectively
Fluent English communication skills at a B2+ level

Senior Data Engineer (Palantir)

Office in Spain: Málaga

& 1 others

Data Software Engineering

Do you have a software engineering background and strong knowledge of Palantir Foundry? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Senior Data Engineer . EPAM is shaping the digital future for Fortune 1000 companies, building complex solutions using modern technologies. We are looking for a Senior Data Engineer with Palantir experience and an open-minded personality who can join our friendly environment and become a core contributor to our team of experts.

responsibilities

Data ingestion of many sources from different domains into Palantir Foundry platform
Very large pipeline Big Data pipeline (PySpark) consisting of more than 350 datasets, together with its orchestration and monitorization
Entity resolution (fuzzy matching) of sites (health care centres) and investigators from many different sources
Data Science: Segmentation and targeting of sites and investigators to let the users to select the best potential ones across different features
Creation of front-end applications to facilitate the consumption of the data by end-users. This uses Workshop, along with coding in TypeScript

requirements

4+ years of front-end software engineering experience
Hands on with Palantir Foundry experience with code workspace, pipeline builder & ontology for back-end pipeline development with Python, PySpark and Spark experience
GenAI experience
Experience with JavaScript, Workshop, Slate

nice to have

Ontology

Senior Data Software Engineer (Databricks)

Remote in Spain

Data Software Engineering

We are looking for a Senior Data Software Engineer with a strong background in Databricks to join our team. The ideal candidate will be an open-minded professional, well-versed in data engineering, who can effortlessly integrate into our friendly work environment and contribute significantly to our projects.

responsibilities

Design, develop, and maintain scalable data pipelines and architectures
Optimize data models and ETL processes using Databricks and other relevant technologies
Implement data quality checks and monitoring to ensure high data integrity
Stay abreast of emerging trends and technologies in data engineering, proposing new tools where beneficial
Troubleshoot and resolve data-related issues promptly
Participate in code reviews to maintain high standards of code quality
Collaborate with other team members and stakeholders to enhance data-driven decisions
Mentor junior data engineers and share knowledge within the team
Contribute to the continuous improvement of our data engineering practices and procedures

requirements

Hands-on experience with Databricks: Delta Lake, workflows, Delta Live Tables, deployment, and versioning
Solid understanding of data architectures and strong data modeling skills
Proficiency in Spark (either Scala or PySpark)
Expertise in designing and building ETL pipelines with Databricks using external orchestrators like Airflow
Experience with cloud-native technologies, containers, and software engineering best practices including unit tests, linting, and code style checks
Engineering background with either AWS, Azure, or GCP
Familiarity with big data and performance optimization of data-intensive applications
Proactivity and experience in client-facing roles
Ability to work independently in ambiguous situations
Desire to function effectively in a transparent, fast-moving startup environment
Fluent English communication skills at a B2+ level

Principal Data Software Engineer (Databricks)

Remote in Spain

Data Software Engineering

We are looking for a Principal Data Software Engineer who possesses expertise in Databricks and has a proactive, open-minded approach. This role offers the opportunity to become a pivotal member of our expert team, making significant contributions in a collaborative environment. Do you have background and wide experience in Data engineering and strong knowledge in Databricks? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Principal Data Software Engineer.

responsibilities

Design, develop, and maintain scalable data pipelines and robust data architectures
Optimize data models and ETL processes using Databricks and complementary technologies
Implement data quality checks and monitoring systems to maintain high data integrity
Stay current with emerging trends and technologies in data engineering, and advocate for the integration of new tools
Troubleshoot and resolve data-related issues efficiently
Conduct code reviews to ensure high standards of code quality are maintained
Drive technology initiatives, including designs, proof of concept, and research and development
Streamline project processes to enhance data engineering practices
Foster transparent and effective communication with team members and clients to justify and discuss technical solutions

requirements

Proven expertise in Spark, using either Scala or PySpark
Strong background in data architecture, data modeling, and building ETL pipelines using Databricks
Experience with multiple SDLC phases and technical leadership over complex implementations
Proficiency in cloud-native technologies and software engineering best practices including unit testing and linting
Engineering background in at least one major cloud platform: AWS, Azure, or GCP
Skillful in performance optimization for data-intensive applications
Keen on technological advancements and modernizing legacy systems
Proven ability to present and advocate for technical solutions to stakeholders
Independent problem-solving skills and comfort with ambiguity
Highly proactive with evident client-facing experience
Fluent English communication skills, minimum B2+ level

Principal Data Software Engineer (Databricks)

Office in Spain: Málaga

& 1 others

Data Software Engineering

We are looking for a Principal Data Software Engineer with experience and knowledge in Databricks and open-minded personality, who can join our friendly environment and become a core contributor to our team of experts. Do you have background and wide experience in Data engineering and strong knowledge in Databricks? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Principal Data Software Engineer.

responsibilities

Design, develop, and maintain scalable data pipelines and robust data architectures
Optimize data models and ETL processes specifically using Databricks and complementary technologies
Implement rigorous data quality checks and monitoring systems to uphold high data integrity
Keep abreast of emerging trends and technologies in data engineering, advocating for the integration of advantageous new tools
Troubleshoot and efficiently resolve data-related issues
Conduct code reviews to ensure adherence to high standards of code quality
Drive technology initiatives, including technology designs, proof of concept, research and development
Streamline project processes when necessary to promote excellence in data engineering practices
Foster transparent, effective communication with team members and clients to justify and discuss technical solutions

requirements

Proven expertise in Spark (either using Scala or PySpark)
Strong background in data architecture and data modeling, along with building ETL pipelines using Databricks
Experience with multiple SDLC phases across various projects, and demonstrated technical leadership over complex implementations
Proficiency in cloud-native technologies including containerization, and software engineering best practices like unit testing and linting
Engineering background with one or more major cloud platforms: AWS, Azure, or GCP
Skillful at performance optimization for data-intensive applications
Keen follower of technological advancements with a knack for modernizing legacy systems
Proven ability to present and advocate for technical solutions to stakeholders
Independent problem solver comfortable with ambiguity
Highly proactive with evident client-facing experience
Fluent English communication skills with a minimum of B2+ level

Senior Data Platform Engineer (Databricks)

Remote in Spain

Data Software Engineering

We are seeking an experienced Senior Data Platform Engineer with a profound expertise in Databricks and multi-cloud environments to join EPAM. In this role, you'll engineer scalable, high-performance data solutions that unlock the full potential of data for our Fortune 1000 clients. If you thrive in building data platforms from scratch and driving innovations across cloud platforms, this opportunity at EPAM could be perfect for you.

responsibilities

Architect and deploy robust, scalable, and secure data platforms using Databricks, ensuring optimal performance and security
Create cloud-agnostic solutions across AWS, Azure, and GCP to ensure flexibility and system resilience
Design and implement end-to-end data pipelines integrating data lakes, warehouses, and streaming frameworks
Leverage Databricks SQL, Delta Lake, MLflow, and advanced Spark optimization techniques for data interaction and performance enhancements
Collaborate with cross-functional teams to implement and maintain workflows based on Databricks following best practices
Develop and maintain CI/CD pipelines specifically tailored for seamless data platform deployment and testing
Set up and manage monitoring, logging, and alerting frameworks to ensure infrastructure health and operational excellence
Optimize compute and storage resources to achieve cost-efficiency without compromising on performance
Troubleshoot and resolve issues related to Databricks and Spark performance
Mentor team members on cluster management, job optimization, and resource allocation within Databricks environments
Ensure adherence to compliance standards and maintain platform security
Drive adoption of advanced capabilities in Databricks like Photon and Graviton instances for improved efficiency
Regularly update and refine existing architectures to meet changing business and technology needs

requirements

Extensive experience with Databricks, Apache Spark, and distributed data processing systems
Strong programming skills in Python, Scala, and SQL
Proficiency in AWS (specifically S3, IAM, Lambda), Azure, or GCP focusing on data engineering services
Expertise in data architecture principles, including data lakes, lakehouses, and ETL workflows
Hands-on experience with CI/CD tools and infrastructure as code practices (Terraform, CloudFormation preferred)
Familiarity with monitoring and observability frameworks suitable for large-scale data environments
Strong analytical and problem-solving skills
Excellent communication and teamwork abilities
Ability to self-manage and operate effectively in a dynamic environment

nice to have

Certifications in Databricks, AWS, Azure, or GCP
Knowledge of Kubernetes and containerized deployments for data pipelines
Experience with real-time data streaming frameworks and governance tools

Senior Data Software Engineer (Databricks)

Office in Spain: Málaga

& 1 others

Data Software Engineering

We are seeking a Senior Data Software Engineer skilled in Databricks to join our team. You will be integral to our efforts, bringing a deep understanding of data engineering, expertise in Databricks, and an open-minded approach to our collaborative and friendly work environment. This role is ideal for a professional eager to develop scalable data solutions and drive innovation in our data practices.

responsibilities

Design, develop, and maintain scalable data pipelines and architectures
Optimize data models and enhance ETL processes utilizing Databricks and related technologies
Implement rigorous data quality checks and monitoring to guarantee high integrity
Keep abreast of the latest trends and technologies in data engineering and advocate for the adoption of beneficial new tools
Troubleshoot and resolve data-related issues promptly
Participate in code reviews ensuring adherence to high standards of code quality

requirements

Hands-on experience with Databricks, including Delta Lake, workflows, Delta Live Tables, along with deployment and versioning strategies
Solid understanding of data architectures and data modeling skills
Expertise in Spark, using either Scala or PySpark
Proficiency in cloud-native technologies and software engineering best practices such as containers, unit tests, linting, and code style checks
Engineering background in AWS, Azure, or GCP
Experience with big data and performance optimization of data-intensive applications
Proactive approach and prior client-facing experience
Ability to navigate ambiguity and work independently
Desire to engage in a transparent, fast-moving startup environment
Fluent English communication skills at a B2+ level or higher

Senior Big Data Software Engineer

Remote in Spain

Data Software Engineering

Do you have a software engineering background and strong knowledge of Big Data? Are you an open-minded professional with good English skills? If it sounds like you, this could be the perfect opportunity to join EPAM as a Senior Big Data Engineer . EPAM is shaping the digital future for Fortune 1000 companies, building complex solutions using modern technologies. We are looking for a Senior Big Data Engineer with an open-minded personality who can join our friendly environment and become a core contributor to our team of experts.

responsibilities

Develop and implement innovative analytical solutions using Cloud Native, Big Data, and NoSQL related technologies
Develop and implement Cloud/On-Premise/Hybrid solutions using best in the class data frameworks
Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
Build collaborative partnerships with architects, technical leads and key individuals within other functional groups
Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution
Actively participate in code review and test solutions to ensure it meets best practice specifications
Build and foster a high performance engineering culture, mentor team members and provide team with the tools and motivation
Write project documentation

requirements

Coding experience with one of the following programming languages: Python/Java/Scala
Experience with Linux OS: configure services and write basic shell scripts, understanding of network fundamentals
Good knowledge of SQL and relational algebra
Advanced experience in software development with Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
Experience building data ingestion pipelines, Data Warehouse or Database architecture
Experience with data modeling; hands-on development experience with modern Big Data components
Cloud: experience in designing, deploying and administering scalable, available and fault tolerant systems
Good understanding of CI/CD principles and best practices
Analytical approach to problem; excellent interpersonal, mentoring and communication skills
Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
Motivated, independent, efficient and able work under pressure with a solid sense for setting priorities
Ability to work in a fast-paced (startup like) agile development environment
Experience in high load and IoT Data Platform architectures and infrastructures
Vast experience with Containers and Resource Management systems: Docker and Kubernetes
Experience in direct customer communications
Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
Experienced in different business domains
English proficiency
Advanced understanding of distributed computing principles

technologies

Python/Java/Scala/Kotlin and SQL
Cloud-Native stack: Databricks, Azure DataFactory, AWS Glue, AWS EMR, Athena, GCP DataProc, GCP DataFlow
Big Data stack: Spark Core, Spark SQL, Spark ML, Kafka, Kafka Connect, Airflow, Nifi, Streamset
NoSQL: CosmosDB, DynamoDB, Cassandra, HBase; MongoDB
Queues and Stream processing: Kafka Streams; Flink; Spark Streaming
Data Visualization: Tableau, PowerBI, Looker
Operation: Cluster operation, Cluster planning
Elasticsearch/ELK
Solid Cloud experience with one of the leading cloud providers (AWS/Azure/GCP): Storage; Compute; Networking; Identity and Security; NoSQL; RDBMS and Cubes; Big Data Processing; Queues and Stream Processing; Serverless; Data Analysis and Visualization; ML as a service (SageMaker; Tensorflow)
Enterprise Design Patterns (Secure Inversion of Control etc)
Development Methods (TDD, BDD, DDD)
Version Control Systems (Git)
Testing: Component/ Integration Testing, Unit testing (JUnit)
Deep understanding of SQL queries, joins, stored procedures, relational schemas, and SQL optimization
Experience in various messaging systems, such as Kafka, RabbitMQ, Event Hub, Pub/Sub
Rest, Thrift, GRPC
Build Systems: Maven, SBT, Ant, Gradle
Docker, Kubernetes

Let us find a perfect job for you

Share your CV and pass our review to get a personalized job offer even if you didn't find a job on the site.

find me a job

Positions covered by Labor Condition Applications pursuant U.S. law can be found here.

Viewing 1-10 out of 60 jobs found

Data Engineer in Spain

FILTERS

Workplace type

City

Open to relocation

Specializations

Skills

Seniority

Other jobs