Reference Agent

An open customer-support agent built and tested to the catalogs, layer by visible layer.

Active · join meAI EngineeringForward-deployed ready

7 of 12 tasks done

1 · Purpose

The Reference Agent is a working customer-support agent for a clothing web shop: order questions, cancellations, returns and refunds. It is built in the open, with no agent framework, so every layer of the harness is visible as its own module. It shows that the two catalogs and AgentTwin are enough to specify, build and test a production-grade agent. Teams can copy its structure, its tests and its four release gates instead of starting from a blank page.

For: Engineers building their first production agent, architects who want a worked example, course participants.

2 · Business benefits

A proven starting pointTeams start from a support agent that already passes behaviour, feature, structure and harness gates.
Money moves only with authorityRefunds above a limit wait for a person, and the agent cannot be talked past that rule.
Answers without paying for a modelCommon questions such as order status are answered by code, which cuts model cost and latency.
Safe to changeAbout 2,100 tests run in four minutes without network or keys, so a change is checked before release.

3 · Status and how to join

Active · join me. Under way: pick a task below, or tell me what you'd like to learn and I'll suggest one. Lead: Basant Choudhary.

Good first tasks this week:

  • Write a scenario for one obligation in NOT_EXERCISED.md
  • Add a paraphrase test case to the router
  • Improve the run-it-yourself steps in the README

4 · Requirements

What it must do

How well it must do it

5 · Architecture, very high level

ChannelReceives customer messages andsends replies.IdentityProves who is asking from acredential the caller cannotRouterAnswers with code when it can;sends the rest to the agentContext assemblyBuilds what the model sees fromthe conversation and facts.Control loopRuns model and tool stepswithin budgets for steps, timeTool layerCalls the shop's systems overMCP, with idempotency keys.Approvals and escalationDurable waits for a person'sdecision or a handoff.Telemetry and checksTraces every turn and runsreply checks inline and online.message with cverified custoprompttool callsrefund over li
Components and what each is responsible for, with no tool names: the tools come in the next section.

Also: Router → Control loop: needs the model · Control loop → Telemetry and checks: trace and reply · Telemetry and checks → Channel: checked reply

6 · Tech stack, per platform

ComponentOpen stackAzureAWS
ChannelChatwoot ✓——
IdentityKeycloak, PyJWT ✓——
Control loophand-rolled Python loop; Groq through a LiteLLM gateway ✓the shared harness library carried to the Azure claims agent ✓ (the clothing agent itself is not deployed to Azure)not built
Tool layerofficial MCP SDK; Saleor as the shop ✓——
Approvals and escalationTemporal ✓——
Telemetry and checksOpenTelemetry, Langfuse, Prometheus, Grafana ✓——

Bold with ✓ = built and running today. Everything else is the planned or possible binding: the components stay the same, only the tools change.

7 · What needs to be done

Phase A: every layer built 3 of 3 done

TaskStatusSizeSkill
All eighteen modules, all sixteen harness layers covereddoneLPython
Strict types, import contract and complexity limits in the suitedoneMSoftware design
Every test tagged with the statement it checksdoneMTesting

Four gates pass 4 of 5 done

TaskStatusSizeSkill
Behaviour: 46 scenarios pass in AgentTwindoneMAI testing
Features: 171 of 171 owed statements exerciseddoneMTesting
Harness: 81 owed capabilities met, accepted or not applicabledoneMAI architecture
Lost parcel in the customer's own words now reaches a persondoneSPython
Exercise the 6 tool-using-agent obligations not yet testedtodoMAI testing

Same agent, different stack 0 of 2 done

TaskStatusSizeSkill
Rebuild the support agent on LangGraph and run the same gatestodoLLangGraph
Rebuild it on the Claude Agent SDK and run the same gatestodoLAgent SDKs

Same specs, different domain 0 of 2 done

TaskStatusSizeSkill
Write the hotel support agent's specificationtodoMSpecification design
Generate the hotel agent from the specs and measure reusetodoLAI engineering

Size: S = an evening or two, M = about a week part-time, L = a few weeks.

8 · Roles

RoleHow manyHours / weekWhat they do
Agent engineer24-6 hours/weekBuilds features and fixes findings, spec first.
Test engineer13-5 hours/weekWrites scenarios and keeps the gates honest.
Platform engineer12-4 hours/weekKeeps the open-stack services running in containers.

9 · What you'll learn

Level-up questions

10 · Links