Back to SearchSenior Data Quality Engineer
Remote in Ukraine
Data Quality Engineering
& 8 others
We are looking for an experienced Senior Data Quality Engineer to join our team.
In this position, you will play a key role in safeguarding the accuracy and dependability of our data engineering workflows and platforms. You will collaborate with various teams to uphold rigorous data quality standards throughout our systems.
Responsibilities
- Create and implement automated testing frameworks and strategies for data engineering workflows
- Integrate solutions with Databricks, PySpark, Scala, Spark SQL, and other relevant technologies
- Develop automated tests to ensure data accuracy, consistency, and reliability across platforms
- Embed automated testing into Azure DevOps CI/CD pipelines for efficient testing, building, and deployment
- Perform performance testing to assess the scalability and efficiency of data processing systems
- Work with data engineers and stakeholders to collect requirements and align testing approaches with business objectives
- Establish monitoring for automated tests and deliver regular updates on test coverage, defects, and data quality concerns
Requirements
- Minimum of 3 years’ experience in Data Quality Engineering or a related discipline
- Comprehensive understanding of automated testing frameworks and data engineering principles
- Hands-on experience with Databricks, PySpark, Scala, and Spark SQL
- Familiarity with Terraform and Azure Event Hubs
- Ability to connect testing frameworks with Azure DevOps and manage source control using Git
- Knowledge of pipeline setup and deployment through Terraform
- Experience using Jira for tracking tasks and issues
- Strong analytical and problem-solving skills for addressing complex data and testing challenges
- Proficient English communication skills, both written and spoken, at B2+ level or higher
Nice to have
- Experience working with Power BI for data visualization
- Knowledge of Azure Data Lake and Spark Streaming
- Background in industries with large-scale or data-intensive environments
- Skill in implementing Infrastructure as Code (IaC) solutions