全职 Senior Data Engineer 工作, 薪水, RiDiK (a Subsidiary Of CLPS. Nasdaq: CLPS) Federal Territory of Kuala Lumpur 公司招聘中 - Ricebowl

Role: Senior Data Engineer

Exp: 5+yrs

Location : Malaysia

Notice Period: Immediate to 15days

Key Skills (Mandato

ry
Migrate data pipelines from existing data acquisition framework to the new GDP data acquisition framew

ork- Configure, develop and deliver data ingestion scripts for loading data into T1 data la

yer- Develop and manage ETL/ELT workflows, ensuring data quality, integrity, and reliabili

ty.- Integrate and automate data quality checks and validation processes within data pipelin

es.- Deploy and manage containerized applications using Docker and orchestrate workloads on Kubernet

es.- Work with modern data lake and warehouse technologies such as Iceb

erg- Implement real-time data streaming solutions using Kaf

ka.- Orchestrate complex workflows using Airfl

ow.- Integrate with data catalog and governance tools such as Datahub and Rang

er.- Collaborate with cross-functional teams to understand business requirements and deliver data solutio

ns.- Ensure security, compliance, and best practices in data management and governan

ce.
Key Skills Requi

red:
- Strong proficiency in Linux, Python, and Shell scrip

ting.- Hands-on experience with Docker, Kubernetes, and container orchestra

tion.- Hands-on experience with Minio and Azure Data Lake Storage (ADLS) using S3 proto

cols.- Experience with Apache Iceberg, Kafka, Airflow, Datahub, Trino, and Ra

nger.- Proficiency in Java for data engineering t

asks.- Solid understanding of data modeling, data warehousing, and big data technolo

gies.
Prior Exper

ience:
- Extensive background in building and maintaining data pipelines/ETL pro

cesses.- Experience in implementing and integrating data quality fram

eworks.- Extensive experience in working on migration projects moving from legacy data pipelines to modernized tec