全职 Sr Data Engineer 工作, 薪水 up to MYR 10,066.8, Asian Technology Solutions Federal Territory 公司招聘中 - Ricebowl

1. Requirement Analysis Solution Design

Collaborate with business and data stakeholders to understand data requirements, transformation rules, and use casesTranslate functional requirements into scalable data engineering and ETL design solutionsDefine approach for data ingestion, transformation, orchestration, and access control

2. Data Engineering Development testing

Design and develop robust ETL pipelines using AWS Glue PySparkImplement complex data transformations including joins, aggregations, conditional logic, and business rule processingBuild reusable, metadata-driven ETL frameworks where applicableOptimize data pipelines for performance, scalability, and reliabilityPerform unit testing and end-to-end validation of data pipelinesImplement data quality checks including integrity, completeness, and reconciliationValidate transformation logic and ensure accuracy of aggregated and processed data

3. Orchestration IntegrationDevelop and manage workflow orchestration using AWS Step Functions and EventBridgeImplement event-driven and schedule-based pipeline execution strategiesEnsure seamless integration between data pipelines, control frameworks, and downstream systems

4. Infrastructure Deployment IaCProvision and manage cloud infrastructure using Terraform Infrastructure as CodeDeploy and configure AWS services including Glue, Lambda, DynamoDB, and orchestration componentsEnsure consistent, repeatable, and scalable deployments aligned with DevOps practices

6. Security GovernanceImplement secure data access controls using IAM and Lake FormationEnsure compliance with data governance policies and role-based access requirementsManage data security, encryption, and access auditing

7. Monitoring, Support Continuous Improvement

Set up monitoring, logging, and alerting mechanisms for data pipelines e.g., SNS, audit logsTroubleshoot data and pipeline issues, ensuring timely resolutionContinuously enhance pipeline performance, reliability, and maintainabilityCollaborate in an agile setup to drive continuous delivery and improvements

Must-Have Skills

· 7-10 years of AWS Data Engineering experience.

· Strong experience in AWS Glue PySpark and data processing on S3

· Proven expertise in ETL development and complex data transformations

· Hands-on experience with AWS Step Functions, EventBridge, and orchestration patterns

· Proficiency in Terraform Infrastructure as Code

· Strong SQL and data modeling skills

· Experience with data validation, reconciliation, and quality frameworks

· Solid understanding of IAM, security, and cloud best practices

________________________________________

Nice-to-Have Skills

· Familiarity with Lake Formation and data governance frameworks

· Experience with DynamoDB or metadata-driven ETL architectures

· Exposure to event-driven architectures

· Knowledge of CICD tools

Pay: RM4,256.39 - RM10,066.76 per month

Work Location: In person