Data Preparation: Cleaning and preprocessing data to make it suitable for analysis, implementing data transformation processes to extract data for the creation and loading of Data Products. Provide strategic recommendations that enhance productivity and business performance
Apply modern data engineering best practices, including modular development, automated testing, orchestration, CI/CD pipelines, version control, and environment promotion.
Data Pipeline Development Design, build, and maintain scalable batch and real-time data pipelines using ETL/ELT patterns. Ingest and integrate structured, semi-structured, and unstructured data from diverse sources, including APIs, databases, files, and event streams.Implement incremental loading, change data capture (CDC), and idempotent pipeline design.