全职 Senior Data Engineer 工作, 薪水, Inter Island Group Selangor 公司招聘中 - Ricebowl

Salary: RM 13,500/month

Employment Type: External Contract

Job Description

We are looking for an experienced Data Engineer to join Automotive Industry and support the development of modern cloud-based data solutions using Microsoft Azure and Azure Databricks.

Responsibilities

• Develop and maintain data pipelines using Azure Databricks, Python, and Spark.
• Manage Databricks workflows, ADLS, Hive Metastore, and compute clusters.
• Develop and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
• Work with Azure Key Vault, Entra ID, and Azure Log Analytics.
• Develop and integrate REST APIs using FastAPI.
• Build and deploy containerized applications using Docker and Azure Container Registry.
• Work with NATS and Redis for messaging and caching.
• Collaborate with project teams using SCRUM.

Requirements

• 7+ years of experience in Data Engineering.
• 7+ years of hands-on experience with Azure Databricks, Python, and Azure DevOps.
• Strong knowledge of Apache Spark and SQL.
• Experience with Microsoft Azure, ADLS, Azure Key Vault, and Entra ID.
• Experience with CI/CD, GitHub/Azure Repos, and YAML pipelines.
• Experience with FastAPI, REST APIs, and Docker.
• Strong understanding of cloud-native data engineering.

Good to Have: Unity Catalog, Medallion Architecture, Pandas/Polars, KQL, PowerShell/Bash, SAFe, NATS, and Redis.