Reference Agent
An open customer-support agent built and tested to the catalogs, layer by visible layer.
1 · Purpose
The Reference Agent is a working customer-support agent for a clothing web shop: order questions, cancellations, returns and refunds. It is built in the open, with no agent framework, so every layer of the harness is visible as its own module. It shows that the two catalogs and AgentTwin are enough to specify, build and test a production-grade agent. Teams can copy its structure, its tests and its four release gates instead of starting from a blank page.
2 · Business benefits
3 · Status and how to join
Good first tasks this week:
- Write a scenario for one obligation in NOT_EXERCISED.md
- Add a paraphrase test case to the router
- Improve the run-it-yourself steps in the README
4 · Requirements
What it must do
- Answers order status, cancellation, return and refund questions for a clothing shop.
- Answers questions the specification can settle with code alone, without calling the model.
- Holds refunds above a limit for a person to approve, and resumes when they decide.
- Hands the conversation to a person when a rule says so, or when the model is unavailable.
- Refuses instructions planted in order notes and other untrusted text.
- Shows each customer only their own orders and conversations.
How well it must do it
- No agent framework: one module per harness layer, with an import contract that enforces the architecture.
- Strict typing, import rules and size limits run inside the test suite, so they cannot be skipped.
- Every test names the specification statement it checks, and a coverage map is generated from those tags.
- Model calls can be recorded and replayed, so the suite runs offline at zero cost.
- Every dependency it does not own sits behind an interface, built only in the composition root.
5 · Architecture, very high level
6 · Tech stack, per platform
| Component | Open stack | Azure | AWS |
|---|---|---|---|
| Channel | Chatwoot ✓ | — | — |
| Identity | Keycloak, PyJWT ✓ | — | — |
| Control loop | hand-rolled Python loop; Groq through a LiteLLM gateway ✓ | the shared harness library carried to the Azure claims agent ✓ (the clothing agent itself is not deployed to Azure) | not built |
| Tool layer | official MCP SDK; Saleor as the shop ✓ | — | — |
| Approvals and escalation | Temporal ✓ | — | — |
| Telemetry and checks | OpenTelemetry, Langfuse, Prometheus, Grafana ✓ | — | — |
7 · What needs to be done
Phase A: every layer built 3 of 3 done
| Task | Status | Size | Skill |
|---|---|---|---|
| All eighteen modules, all sixteen harness layers covered | done | L | Python |
| Strict types, import contract and complexity limits in the suite | done | M | Software design |
| Every test tagged with the statement it checks | done | M | Testing |
Four gates pass 4 of 5 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Behaviour: 46 scenarios pass in AgentTwin | done | M | AI testing |
| Features: 171 of 171 owed statements exercised | done | M | Testing |
| Harness: 81 owed capabilities met, accepted or not applicable | done | M | AI architecture |
| Lost parcel in the customer's own words now reaches a person | done | S | Python |
| Exercise the 6 tool-using-agent obligations not yet tested | todo | M | AI testing |
Same agent, different stack 0 of 2 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Rebuild the support agent on LangGraph and run the same gates | todo | L | LangGraph |
| Rebuild it on the Claude Agent SDK and run the same gates | todo | L | Agent SDKs |
Same specs, different domain 0 of 2 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Write the hotel support agent's specification | todo | M | Specification design |
| Generate the hotel agent from the specs and measure reuse | todo | L | AI engineering |
8 · Roles
| Role | How many | Hours / week | What they do |
|---|---|---|---|
| Agent engineer | 2 | 4-6 hours/week | Builds features and fixes findings, spec first. |
| Test engineer | 1 | 3-5 hours/week | Writes scenarios and keeps the gates honest. |
| Platform engineer | 1 | 2-4 hours/week | Keeps the open-stack services running in containers. |
9 · What you'll learn
- How a production agent is put together, one harness layer at a time.
- Why answering with code first saves money and errors.
- How durable waits let a person approve a refund hours later.
- How to enforce an architecture with tests rather than diagrams.
- How to route a failure to the specification instead of patching the agent.
Level-up questions
- Which questions can your agent answer without calling the model at all?
- What would break if a refund tool were called twice for one request?
- Why must the identity come from a credential and not from the conversation?
- What does an agent framework take over that this agent writes out by hand?