Job Description
Data Warehouse Design & Architecture:
- Work with business users and Data Analysts to gather requirements and turn them into data models.
- Design and implement highly scalable, performant, and cost-effective data warehouse solutions (e.g., Snowflake, BigQuery, Databricks).
- Develop and maintain robust data models (Dimensional modeling, Data Vault, or hybrid approaches) that support complex business analytics.
Data Pipeline Development:
- Build, orchestrate, and optimize highly reliable ELT/ETL pipelines using tools like Python, SQL, dbt, and Apache Airflow.
- Ensure data pipelines are fault-tolerant, scalable, and fully monitored with automated alerting for data quality issues.
- Optimise SQL and pipeline design for cost and performance on ETL Job.
Single Source of Truth (SSOT) Creation:
- Partner with cross-functional business stakeholders to define, standardize, and calculate core business metrics.
- Develop a robust semantic layer to ensure standardized definitions and metrics are exposed consistently across all downstream tools.
- Implement rigorous data quality checks and data contracts to build absolute trust in the enterprise data assets.
- Lead the onboarding of new sources and the migration from legacy systems, using ID mapping, parallel-run reconciliation and controlled cutover to keep downstream consumers stable.
Enterprise Data Agent Development:
- Design and build "Data Agents" (using LLMs, RAG frameworks, and Text-to-SQL technologies) that act as an intelligent interface over the SSOT and enterprise knowledge bases.
- Integrate unstructured enterprise metadata (glossaries, wiki pages, process docs) with structured data models to give the Data Agent full business context.
- Continuously refine prompt engineering, semantic search embeddings, and guardrails to ensure the agent returns accurate, hallucination-free insights.
Job Requirements
- Core Engineering Experience: 3+ years of experience in data engineering, data warehousing, or software engineering.
- Advanced SQL & Python: Expert-level SQL skills for complex transformations and deep proficiency in Python for pipeline development and API integration.
- Data Warehousing & Modeling: Proven experience with modern cloud data warehouses and strong practical knowledge of data modeling methodologies (Kimball, OBT, etc.).
- Transformation & Orchestration: Hands-on experience with dbt (data build tool) for transformation and Airflow/Dagster/Prefect for orchestration.
- AI & LLM Integration (Bonus/Highly Preferred): Experience with or a strong desire to learn GenAI application frameworks (e.g., LangChain, LlamaIndex), Vector Databases (e.g., Pinecone, Milvus), and OpenAI/Anthropic APIs.
- Communication: Exceptional ability to translate complex business logic into scalable data models and explain technical concepts to non-technical stakeholders.
- Engineering Best Practices: Strong advocate for CI/CD, version control (Git), automated testing, and Infrastructure as Code.
- Domain (Bonus): Retail or e-commerce data such as orders, refunds, promotions, loyalty and GST.
- AI-assisted engineering (Bonus): Uses AI coding assistants (e.g. Claude Code) productively, and verifies their output rather than trusting it.