Back to SearchSenior Data Quality Engineer
Remote in Ukraine
Data Quality Engineering
& 8 others
We are looking for an experienced Senior Data Quality Engineer to join our team.
In this position, you will be responsible for upholding the accuracy and dependability of our data engineering systems. You will collaborate with different teams to ensure consistent and high-quality data across all platforms.
Responsibilities
- Develop and implement automated testing frameworks and strategies for data engineering workflows
- Work with technologies including Databricks, PySpark, Scala, and Spark SQL
- Create automated tests to validate data accuracy, consistency, and reliability across systems
- Integrate automated testing into Azure DevOps CI/CD pipelines for efficient testing, building, and deployment
- Perform performance testing to assess the scalability and effectiveness of data processing solutions
- Partner with data engineers and stakeholders to collect requirements and align testing approaches with business objectives
- Establish monitoring for automated tests and deliver regular updates on test coverage, defects, and data quality metrics
Requirements
- Minimum of 3 years’ experience in Data Quality Engineering or a related discipline
- Comprehensive understanding of automated testing frameworks and data engineering principles
- Hands-on experience with Databricks, PySpark, Scala, and Spark SQL
- Knowledge of Terraform and Azure Event Hubs
- Ability to connect testing frameworks with Azure DevOps and manage source control using Git
- Experience with pipeline configuration and deployment through Terraform
- Familiarity with Jira for managing tasks and issues
- Strong analytical and troubleshooting skills for resolving complex data and testing challenges
- Proficient English communication skills, both written and spoken, at B2+ level or higher
Nice to have
- Experience using Power BI for data visualization
- Familiarity with Azure Data Lake and Spark Streaming
- Background in industries with large-scale or data-intensive environments
- Skill in implementing Infrastructure as Code (IaC) solutions