Morgan McKinley Hiring! Full Time Senior Software Engineer (AI - LLMOps) in - Ricebowl

Job Title: Senior Software Engineer (AI Platform & LLMOps)

Job Duration: 6 months contract with a possibility to extend or to go perm!

Location: One North, Singapore (100% On-site)

Schedule: Monday – Friday, 10:00 AM – 7:00 PM (No Overtime)

The Role

We are seeking a Senior Software Engineer to own our ACS embedded-evaluation workstream. You will embed with AI product teams to build the infrastructure, automated pipelines, and observability tools that ensure our LLM agents perform accurately, cost-effectively, and reliably at scale.

Key Responsibilities

• AI Evaluation Infrastructure: Own and scale EvalsHub (backend, SDK, CLI, frontend). Build golden datasets, regression pipelines, and CI/CD model-release integration.
• Advanced Automation: Convert human review procedures into LLM-as-judge, multi-turn, trajectory, and deterministic evaluators to eliminate manual testing.
• Observability & Tracing: Instrument distributed agent systems using OpenTelemetry across Go/Python services and Temporal workflows to ensure flawless debugging data.
• Optimization: Run experiments across model architectures, RAG, and embeddings to balance accuracy, latency, and cloud costs.
• Enablement: Author technical docs, lead tool migrations, and participate in the LLMOps production operational rotation.

️ Requirements

• Experience: Minimum 2+ years of relevant production experience.
• AI & Workflow Ecosystem: Hands-on with LangSmith, LangGraph/LangChain, FastAPI, or Temporal.
• AI Domain Expertise: Proven track record in LLM-as-judge frameworks, agent-trajectory analysis, RAG, semantic retrieval, or Text-to-SQL.
• Data & Core Tech: Strong analytical skills handling geospatial data. Proficiency in React/TypeScript (for developer tools), Grafana, and Redis.
• Soft Skills: Metrics-driven problem solver with excellent cross-functional communication skills.