Build and operate monitoring and observability for ML systems (data drift, model performance, system health), and define alerting/incident response processes.
Build and manage Azure CaaS solutions using Azure Kubernetes Service (AKS), Azure Container Instances (ACI), or other container-based services to support scalable workloads.
· Good knowledge of Python programming and automation is an advantage, particularly for utilities, integration, testing, and producer/consumer development.
The role is for a self-driven senior architect with deep VDI and endpoint expertise. The candidate will be extensively involved in hands-on activities including design, documentation, standards definition and design governance. Candidate must be comfortable working with senior stakeholders and across a geographically distributed global team.
Source Integration and Incremental Processing: Implement reliable ingestion from databases, files, APIs and event streams using full-load, incremental, change-data-capture and replayable processing patterns appropriate to each source.
Data Pipeline Architecture and Delivery: Lead the design and implementation of maintainable batch and streaming pipelines that ingest, validate, transform and publish structured, semi-structured and unstructured data across AWS and Databricks.
Technical Programming Skills: Proficient in Python programming language with experience in data manipulation libraries (pandas, numpy), data pipeline frameworks, and API development. Strong SQL skills for data querying, transformation, and database management across various database platforms.