Back to SearchSenior Data Quality Engineer
Remote in Ukraine
Data Quality Engineering
& 8 others
We are seeking a skilled Senior Data Quality Engineer to join our team.
In this role, you will be responsible for maintaining the integrity and reliability of our data engineering pipelines and platforms. You will work closely with cross-functional teams to ensure high standards of data quality across all systems.
Responsibilities
- Design and build automated testing frameworks and strategies for data engineering pipelines
- Focus on integration with Databricks, PySpark, Scala, Spark SQL, and related technologies
- Develop and implement automated tests to validate data integrity, quality, and reliability across all data platforms
- Integrate automated testing processes within Azure DevOps CI/CD pipelines for streamlined testing, building, and deployment
- Conduct performance testing to evaluate the efficiency and scalability of data processing systems
- Collaborate with data engineers and stakeholders to gather requirements and align testing strategies with business needs
- Set up monitoring for automated tests and provide regular reports on test coverage, defects, and data quality issues
Requirements
- At least 3 years of experience in Data Quality Engineering or a similar field
- Strong understanding of automated testing frameworks and data engineering concepts
- Proficiency with Databricks, PySpark, Scala, and Spark SQL
- Experience with Terraform and Azure Event Hubs
- Ability to integrate testing frameworks with Azure DevOps and manage source control using Git
- Knowledge of pipeline configuration and deployment using Terraform
- Familiarity with Jira for task and issue tracking
- Strong analytical and troubleshooting skills for resolving complex data and testing issues
- Fluent English communication skills, both written and spoken, at B2+ level or higher
Nice to have
- Experience with Power BI for data visualization
- Familiarity with Azure Data Lake and Spark Streaming
- Background in data-intensive industries or large-scale data environments
- Proficiency in implementing solutions using Infrastructure as Code (IaC)