Senior Data Engineer

Redesign the database to enhance the existing way of storing and sharing the data

Redesign of Database web layer to enhance the manual process for Client service of survey creation

Setup data acquisition on aggregated and respondent levels from KAP, Affectiva, Context Lab,LinkDB, Link+DB

Create a calculation module (study bank) in EDP which will conduct new calculations using reference data

Create API for EDP to pass the data to other systems (e.g. Dashboard)

Ensure new calculations and norms are passed to EDP for storage

Migrate historical data from LinkDB and CXL

3+ years of experience in Data Engineering

Strong knowledge of Apache Kafka

Strong knowledge of Azure Data Factory

Strong knowledge of Azure Data Lake Storage

Strong knowledge of Databricks

Strong knowledge of PySpark

Strong knowledge of Python

Strong knowledge of SQL