全职 Data Engineer 工作, 薪水, Skill Quotient Federal Territory 公司招聘中 - Ricebowl

Role Summary:

You’ll be a central part of our team evolving, developing, and maintaining the data engineering and ingestion pipelines within our enterprise data ecosystem.

We have a modern cloud-based data platform providing technology that supports the collection, processing, and consumption of data from all business units, using batch or streaming patterns and making it available to analysts, data scientists, and machine learning models.

Our platform is being migrated from Azure to AWS, we need to recreate/ revamp the pipelines with the new platform.

Key Responsibilities

• Developing and maintaining production-grade and peer-reviewed data ingestion pipelines.
• Ensuring that data is made available in a way that is optimised for security, stability, and performance
• Collaborating with the peer data engineers, architects and other stakeholders to identify and solve problems within data engineering practices.
• Participating in developing the capabilities and governance of the data platform
• Setting up monitoring and orchestration of data pipelines.

Technical / Functional Skills:

Professional qualifications:

• Have professional data engineering experience in data analytics, data processing tools, ETL, ELT etc.
• Have strong hands-on Python and SQL skills with DevOps and CI/CD (GitHub Actions, Azure DevOps etc.) knowledge.
• Have experience working with Spark/PySpark, Kafka or NiFi.
• Are familiar with cloud platforms (AWS, Azure) and container technology (Kubernetes, Docker)

Personal qualifications:

• Good collaborating and communication skills for engaging with stakeholders at all levels
• Have an experimental mindset and appreciate learning new things
• Are curious and engaged with a passio