全职 Data Engineer 工作, 薪水 up to MYR 8,500, Tech-matrix Federal Territory 公司招聘中 - Ricebowl

Job Title: Senior Data Engineer
Experience: 6+ Years
Location: Kuala Lumpur, Malaysia (Onsite/Hybrid)
Employment Type: Contract(Long Term)

Job Summary

We are looking for an experienced Senior Data Engineer with 6+ years of hands-on experience in designing, developing, and optimizing enterprise-scale data engineering solutions. The ideal candidate should have strong expertise in PySpark, Azure Data Platform, SQL, Python, Kafka, ETL development, and Cloud Data Engineering, with experience building real-time and batch data pipelines.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build high-performance data processing solutions using PySpark and Python.
• Develop and maintain data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Work with Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Blob Storage.
• Build and optimize real-time streaming pipelines using Apache Kafka and Kafka Connect.
• Develop Change Data Capture (CDC) solutions using Debezium and Kafka Connectors.
• Create optimized SQL queries, stored procedures, and data transformation logic.
• Design dimensional data models, star schema, snowflake schema, and data marts.
• Integrate on-premise data sources with Azure and AWS cloud platforms.
• Work with cloud-based ETL services such as AWS Glue and AWS Athena.
• Optimize Spark jobs using partitioning, caching, and performance tuning techniques.
• Develop CI/CD pipelines for deployment and automation using Git and DevOps tools.
• Monitor and troubleshoot data pipelines using Azure Monitor and logging tools.
• Collaborate with business analysts, architects, developers, and QA teams to deliver enterprise data solutions.
• Prepare technical documentation and maintain data architecture documentation.
• Follow Agile/Scrum development methodologies.

Required SkillsMandatory

• 6+ years of experience in Data Engineering.
• Strong experience in PySpark.
• Strong programming skills in Python.
• Expert-level SQL development and performance tuning.
• Hands-on experience with Azure Data Factory (ADF).
• Experience with Azure Databricks.
• Experience working with Azure Data Lake Storage (ADLS).
• Experience with Azure Synapse Analytics.
• Experience building ETL/ELT pipelines.
• Experience with Apache Kafka.
• Strong knowledge of data warehousing concepts.
• Experience in batch and real-time data processing.
• Experience with Git and CI/CD deployment.
• Strong analytical and problem-solving skills.

Pay: From RM8,500.00 per month

Benefits:

• Health insurance
• Opportunities for promotion
• Professional development

Work Location: In person