cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Core loop › Context assembly › Trimming

Agent Harness Tutorial · End to end · Zoom level 3

The budget and trimming

The model can read a lot, but more is slower, dearer, and easier to confuse. So each call has a budget of 24,000 characters (roughly 8,000 tokens). If the context is bigger, whole exchanges are dropped from the middle.

systemfirst exchangeexchange 2exchange 3dropped firstexchange 4exchange 5latestALWAYS KEPTALWAYS KEPTwhile there are more than 2 exchanges and it is over 24,000 characters: drop the middle onea tool call and its results count as ONE exchange, so they are never split
The first exchange sets up the task and the latest is what is being answered, so both stay. The middle goes first.

Why the middle? The beginning says what the conversation is about. The end is what Rohan just asked. The middle is the part least needed right now.

Why whole exchanges? A tool request without its result, or a result without its request, confuses the model and some providers reject it outright. After trimming, a final check makes sure no call has lost its answer; if one has, the turn fails rather than send a broken transcript.

Trimming works on a copy, for one call only. The saved conversation has its own, separate limit of 32,000 characters.

Known gaps, written down24,000 is a target, not a hard ceiling: with only two exchanges left nothing more is dropped, so eight big tool results could all be sent. And the newest question is a separate unit from the tool call it caused, so in rare cases it could be trimmed away. The planned fix: always protect the latest message and everything after it. The code counts characters, not tokens; a real token counter (tiktoken) is on the list.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

agent_harness/context/__init__.py:143Drop whole exchanges from the middle until it fits
def assembled(
    *,
    system: str,
    history: Sequence[Message],
    max_chars: int = 24_000,
) -> Assembly:
    """Order for cache friendliness, then trim whole exchanges from the middle.

    The system prompt is a stable prefix and goes first and unchanged: any byte
    that moves invalidates every cached token after it. Trimming takes from the
    middle rather than the tail, because the most recent turns are what the model
    is answering and the earliest establish the task.

    **Trimming drops whole exchanges.** An earlier version dropped individual
    messages by index and orphaned a tool call in eight of fifty-five
    turn-and-budget combinations — an assistant turn claiming a call whose answer
    had been removed, which every provider rejects. Nothing caught it because no
    test ran a conversation long enough to trim.

    Compaction is still deliberately absent: a summary inherits the provenance of
    everything it summarised, and getting that wrong is worse than a shorter
    window.
    """
    head = Message(role="system", content=system, provenance="operator")
    units = exchanges(history)
    before = len(units)

    def size(groups: Iterable[list[Message]]) -> int:
        return sum(len(m.content) for group in groups for m in group)

    while len(units) > 2 and size(units) + len(system) > max_chars:
        units.pop(len(units) // 2)

    kept = [m for group in units for m in group]