Back to SearchSenior Data Engineer
Remote in Colombia
Data Software Engineering
& 4 others
Choose an option
We are searching for a highly skilled and experienced Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for data onboarding and costumes integration projects. You must possess excellent AWS infrastructure-building expertise, including integration building with API, FTP, or JDBC connectors. Your thorough knowledge of infrastructure as code and git/CI/CD processes will be highly valued in this role.
Responsibilities
- Collaborate with the team to deliver scalable, robust, and cost-effective data products and solutions
- Develop AWS Cloud infrastructure including data pipelines and connectors
- Produce data in the data lake through AWS Glue and Lake formation
- Establish unit and data tests for the onboarding process
- Manage the data architecture and make recommendations for improvements
Requirements
- 3+ years of experience building data pipelines in Python, with PySpark experience
- Strong understanding of AWS Cloud fundamentals
- Solid knowledge in infrastructure as code using CDK
- Git & CI/CD Knowledge is a must
- Experience with common data Python libraries such as Pandas and AWS Wrangler
- Understanding of REST APIs from the consumer perspective
- ML knowledge is a plus, particularly with Kubeflow