Agent Harness Tutorial · End to end · Zoom level 3
The budget and trimming
The model can read a lot, but more is slower, dearer, and easier to confuse. So each call has a budget of 24,000 characters (roughly 8,000 tokens). If the context is bigger, whole exchanges are dropped from the middle.
Why the middle? The beginning says what the conversation is about. The end is what Rohan just asked. The middle is the part least needed right now.
Why whole exchanges? A tool request without its result, or a result without its request, confuses the model and some providers reject it outright. After trimming, a final check makes sure no call has lost its answer; if one has, the turn fails rather than send a broken transcript.
Trimming works on a copy, for one call only. The saved conversation has its own, separate limit of 32,000 characters.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
agent_harness/context/__init__.py:143Drop whole exchanges from the middle until it fitsdef assembled(
*,
system: str,
history: Sequence[Message],
max_chars: int = 24_000,
) -> Assembly:
"""Order for cache friendliness, then trim whole exchanges from the middle.
The system prompt is a stable prefix and goes first and unchanged: any byte
that moves invalidates every cached token after it. Trimming takes from the
middle rather than the tail, because the most recent turns are what the model
is answering and the earliest establish the task.
**Trimming drops whole exchanges.** An earlier version dropped individual
messages by index and orphaned a tool call in eight of fifty-five
turn-and-budget combinations — an assistant turn claiming a call whose answer
had been removed, which every provider rejects. Nothing caught it because no
test ran a conversation long enough to trim.
Compaction is still deliberately absent: a summary inherits the provenance of
everything it summarised, and getting that wrong is worse than a shorter
window.
"""
head = Message(role="system", content=system, provenance="operator")
units = exchanges(history)
before = len(units)
def size(groups: Iterable[list[Message]]) -> int:
return sum(len(m.content) for group in groups for m in group)
while len(units) > 2 and size(units) + len(system) > max_chars:
units.pop(len(units) // 2)
kept = [m for group in units for m in group]