Data Platform Engineer
Working location: KL area
Cloud Infrastructure & Platform Engineering
- Design, deploy, and manage scalable Azure infrastructure supporting Data Platform workloads, including Azure VMs, Synapse Analytics, Data Lake, Lakhouse, Databricks, Fabric and related services.
- Implement Infrastructure as Code (IaC) using tools such as Terraform templates to ensure consistent, repeatable deployments.
- Develop and maintain CI/CD pipelines (Azure DevOps or equivalent) to automate provisioning, configuration, and deployment of Azure resources.
- Support data platform operations, enabling reliable data ingestion, transformation, and analytics workflows.
- Assist in migrating on-premise or legacy workloads to Azure, ensuring minimal disruption and adherence to best practices.
- Monitor platform performance, availability, and reliability using Azure monitoring and observability tools.
- Manage Azure resource lifecycle, scalability, and cost optimization, including budgeting, usage analysis, and resource rightsizing.
Cloud Security & DevSecOps
- Implement and maintain Azure security controls aligned with organizational and industry best practices.
- Configure and manage identity and access controls using Azure Active Directory, RBAC, Managed Identities, and Privileged Identity Management.
- Support secure network architecture using Azure networking services (VNets, NSGs, Private Endpoints, Firewalls, etc.).
- Implement data protection measures including encryption at rest and in transit, secure secrets management (Key Vault), and secure service access.
- Integrate security practices into CI/CD pipelines, including secure configuration, credential protection, and automated security checks where applicable.
- Monitor cloud environment for security events, misconfigurations, and vulnerabilities using Azure-native security tools (e.g., Defender for Cloud).
- Support incident response activities, including investigation, containment, and remediation of infrastructure and security incidents.
- Support compliance and security clearance processes in coordination with Group/HQ Security teams, ensuring adherence to organizational security policies, legal regulations, and PII data protection requirements.
Operations, Monitoring, and Continuous Improvement
- Participate in incident management, troubleshooting, and system recovery to ensure high platform availability.
- Maintain documentation for infrastructure architecture, deployment procedures, security configurations, and operational processes.
- Continuously improve platform reliability, automation, and security posture through proactive enhancements.
- Stay up to date with Azure services, DevOps practices, and cloud security development