Back to SearchSenior Data Quality Engineer
Remote in Ukraine
Data Quality Engineering
& 8 others
We are seeking a Senior Data Quality Engineer with a strong background in data engineering to join our team.
In this role, you will be responsible for ensuring the reliability and precision of our data systems. You will work closely with various teams to maintain high standards of data quality throughout all platforms.
Responsibilities
- Design and deploy automated testing frameworks and strategies for data engineering processes
- Utilize technologies such as Databricks, PySpark, Scala, and Spark SQL
- Build automated tests to confirm data accuracy, consistency, and reliability across platforms
- Integrate automated testing into Azure DevOps CI/CD pipelines for streamlined testing, building, and deployment
- Conduct performance testing to evaluate the scalability and efficiency of data processing systems
- Collaborate with data engineers and stakeholders to gather requirements and align testing strategies with business needs
- Set up monitoring for automated tests and provide regular updates on test coverage, defects, and data quality metrics
Requirements
- At least 3 years of experience in Data Quality Engineering or a related field
- Thorough understanding of automated testing frameworks and data engineering concepts
- Practical experience with Databricks, PySpark, Scala, and Spark SQL
- Knowledge of Terraform and Azure Event Hubs
- Ability to integrate testing frameworks with Azure DevOps and manage source control using Git
- Experience with pipeline configuration and deployment using Terraform
- Familiarity with Jira for task and issue management
- Strong analytical and troubleshooting skills for complex data and testing challenges
- Effective English communication skills, both written and spoken, at B2+ level or higher
Nice to have
- Experience with Power BI for data visualization
- Familiarity with Azure Data Lake and Spark Streaming
- Background in large-scale or data-intensive industries
- Ability to implement Infrastructure as Code (IaC) solutions