- North Bridge Road Downtown Core Central Region (Singapore) Singapore
全职 Data Engineer 工作, 薪水 up to SGD 9,000, UNISYNC SYSTEMS PTE. LTD. Central Region (Singapore) 公司招聘中 - Ricebowl
分享
保存
工作地点
职位描述
岗位职责
Key Responsibilities
Design, develop and maintain scalable data pipelines and data ingestion frameworks for large-volume datasets.
Develop data transformation and processing applications using Apache Spark, PySpark, Scala and Python.
Build and optimize data pipelines using Azure Databricks, Azure Data Factory, AWS EMR and related cloud services.
Work with Hadoop, HDFS, Hive, Snowflake, Teradata and Data Lake environments.
Develop batch and real-time data processing solutions using Spark Structured Streaming and Kafka.
Perform data extraction, transformation and loading across heterogeneous source and target systems.
Develop and optimize Spark SQL, HiveQL and SQL queries for performance and cost efficiency.
Design data models, partitioning strategies and scalable data storage architectures.
Build and manage workflow orchestration using Apache Airflow.
Implement CI/CD pipelines and automated testing using tools such as Jenkins, Docker, GitHub Actions and pytest.
Troubleshoot data pipeline, performance and production issues and implement sustainable solutions.
Collaborate with business stakeholders, architects and technology teams to understand requirements and deliver data engineering solutions.
Ensure data quality, reliability, security and operational stability across enterprise data platforms.
Required Skills
6+ years of experience in Data Engineering / Big Data Engineering.
Strong hands-on experience with Apache Spark / PySpark.
Strong programming skills in Python and/or Scala.
Good experience with Hadoop, HDFS and Hive.
Experience developing ETL/ELT and data ingestion pipelines.
Strong SQL and data processing skills.
Experience with Azure Databricks, Azure Data Factory, AWS EMR or equivalent cloud data platforms.
Experience with Kafka / real-time streaming is an advantage.
Hands-on experience with Airflow and data pipeline orchestration.
Experience with Snowflake, Teradata, SQL Server or other enterprise databases.
Good understanding of Data Lake, Delta Lake, Data Warehousing and Data Modelling.
Experience with Git, CI/CD, Docker and automated testing.
Design, develop and maintain scalable data pipelines and data ingestion frameworks for large-volume datasets.
Develop data transformation and processing applications using Apache Spark, PySpark, Scala and Python.
Build and optimize data pipelines using Azure Databricks, Azure Data Factory, AWS EMR and related cloud services.
Work with Hadoop, HDFS, Hive, Snowflake, Teradata and Data Lake environments.
Develop batch and real-time data processing solutions using Spark Structured Streaming and Kafka.
Perform data extraction, transformation and loading across heterogeneous source and target systems.
Develop and optimize Spark SQL, HiveQL and SQL queries for performance and cost efficiency.
Design data models, partitioning strategies and scalable data storage architectures.
Build and manage workflow orchestration using Apache Airflow.
Implement CI/CD pipelines and automated testing using tools such as Jenkins, Docker, GitHub Actions and pytest.
Troubleshoot data pipeline, performance and production issues and implement sustainable solutions.
Collaborate with business stakeholders, architects and technology teams to understand requirements and deliver data engineering solutions.
Ensure data quality, reliability, security and operational stability across enterprise data platforms.
Required Skills
6+ years of experience in Data Engineering / Big Data Engineering.
Strong hands-on experience with Apache Spark / PySpark.
Strong programming skills in Python and/or Scala.
Good experience with Hadoop, HDFS and Hive.
Experience developing ETL/ELT and data ingestion pipelines.
Strong SQL and data processing skills.
Experience with Azure Databricks, Azure Data Factory, AWS EMR or equivalent cloud data platforms.
Experience with Kafka / real-time streaming is an advantage.
Hands-on experience with Airflow and data pipeline orchestration.
Experience with Snowflake, Teradata, SQL Server or other enterprise databases.
Good understanding of Data Lake, Delta Lake, Data Warehousing and Data Modelling.
Experience with Git, CI/CD, Docker and automated testing.
重要安全守则
申请工作时，切勿提供您的银行或信用卡详细资料。不要转账或完成无关的在线调查问卷。如果您发现可疑内容，请举报此招聘广告。
了解更多