Asian Technology Solutions

KL City

Design and develop robust ETL pipelines using AWS Glue PySparkImplement complex data transformations including joins, aggregations, conditional logic, and business rule processingBuild reusable, metadata-driven ETL frameworks where applicableOptimize data pipelines for performance, scalability, and reliabilityPerform unit testing and end-to-end validation of data pipelinesImplement data quality checks including integrity, completeness, and reconciliationValidate transformation logic and ensure accuracy of aggregated and processed data
Orchestration IntegrationDevelop and manage workflow orchestration using AWS Step Functions and EventBridgeImplement event-driven and schedule-based pipeline execution strategiesEnsure seamless integration between data pipelines, control frameworks, and downstream systems
Infrastructure Deployment IaCProvision and manage cloud infrastructure using Terraform Infrastructure as CodeDeploy and configure AWS services including Glue, Lambda, DynamoDB, and orchestration componentsEnsure consistent, repeatable, and scalable deployments aligned with DevOps practices ...