Agent Harness Tutorial · End to end · Zoom level 2
Model call
The loop calls the model through one tiny socket with one method: send a request, get a response. Everything that can go wrong on the way (rate limits, timeouts, a blocked prompt) is turned into a clear, named result before it reaches the loop.
What goes out. The model name, temperature 0 (so it is as repeatable as possible), at most 4,096 tokens back, the messages from context assembly, and the 8 tools Rohan may use, each with a name, a description and the shape of its inputs.
What comes back. Some text, or some tool requests, or both; the tokens used; and the stop reason. If the stop reason is "length" (the model ran out of room), the answer is thrown away, never sent on half-finished.
What it costs. Every response is priced at once: $0.15 per million tokens in, $0.75 per million out. If the turn reaches $0.50 it stops. A model with no price fails at start-up; it is never priced as free.
Code view: input, process, output, the call stack, the classes
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
agent_harness/contracts/protocols.py:30The model socket: one method@runtime_checkable
class LLMClient(Protocol):
"""AHC-0022 — provider interaction is substitutable without changing the
system. One choke point, every call passes through it."""
async def complete(self, request: ModelRequest) -> ModelResponse: ...agent_harness/contracts/model.py:79What goes outclass ModelRequest(BaseModel):
model_config = ConfigDict(frozen=True)
messages: tuple[Message, ...]
tools: tuple[dict[str, object], ...] = ()
max_tokens: int = 4096
temperature: float | None = None
"""None takes the client's configured temperature, which is zero by default:
determinism is the floor, variance is opted into. A number is this call's
own. It was `0.0` meaning *unset*, so asking for zero under a non-zero
configuration was read as not asking (F-064)."""agent_harness/contracts/model.py:92What comes backclass ModelResponse(BaseModel):
model_config = ConfigDict(frozen=True)
text: str = ""
tool_calls: tuple[ToolCall, ...] = ()
usage: Usage = Field(default_factory=Usage)
model: str = ""
stop_reason: str = ""
gateway: dict[str, str] = Field(default_factory=dict)
"""What a gateway in front of the provider said about this call, by
lower-cased header name (`x-content-safety: pass; prompt=pass; …`). Empty
when there is no gateway or it said nothing (`llm.gateway.SIGNALS`)."""
@property
def wants_tools(self) -> bool:
return len(self.tool_calls) > 0config/azure.yaml:32Which plug fills the socket on Azure model:
adapter: apim-ai-gateway
base_url: {env: APIM_GROQ_BASE_URL}
api_key: {key_vault: apim-subscription-key}Tech choices (what we chose, Azure and open-source options): L2 · Model invocation