Назад до пошукуSenior Data Quality Engineer
Віддалений* формат співпраці з території - Україна
*Ви можете працювати віддалено з території країни (або країн), для яких відкрита ця позиція.
Data Quality Engineering& інші навички
Не знайшли свою вакансію?
Поділіться своїм CV — ми запропонуємо роль, що відповідає вашим навичкам та досвіду.
Знайти мою роль →We are seeking a skilled Senior Data Quality Engineer to join our team. The successful candidate will play a crucial role in ensuring the integrity, reliability, and quality of our data engineering pipelines and platforms.
Чим ви будете займатися у цій ролі
- Design and develop a robust automated testing framework and strategy for data engineering pipelines, with a focus on integration with Databricks, PySpark, Scala, Spark SQL, and other relevant technologies
- Implement automated tests to ensure data integrity, quality, and reliability across all data pipelines and platforms
- Integrate automated testing within Azure DevOps CI/CD pipelines to ensure seamless testing, build, and deployment processes
- Conduct and execute performance tests to assess the efficiency and scalability of data processing systems
- Work closely with data engineers and other stakeholders to understand requirements and ensure testing strategies are aligned with business objectives
- Establish monitoring for automated tests, providing regular reports on test coverage, defects, and data quality issues
Навички
- Minimum of 3 years of experience in Data Quality Engineering or a related field
- Solid understanding of automated testing frameworks and data engineering principles
- Proficiency with Databricks, PySpark, Scala, Spark SQL, Terraform, and Azure Event Hubs. Solid understanding of automated testing frameworks and data engineering principles
- Experience in integrating testing frameworks with Azure DevOps and managing source control with Git
- Knowledge of pipeline configuration and deployments utilizing Terraform
- Familiarity with Jira
- Strong analytical abilities with a proven track record in troubleshooting and resolving complex data and testing challenges
- Fluency in English with a B2+ level
Буде перевагою
- Experience with Power BI, Azure Data Lake, Spark Streaming, and other data processing and visualization tools
- Prior experience in a data-intensive industry or with large-scale data environments
- Proficient in implementing solutions through Infrastructure as Code (IaC)