cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 7 · After the loop › promise gate

Agent Harness Tutorial · End to end · Zoom level 3

How we take care of it: promise gate

promise gate catches "we'll call you" with nothing behind it.

Open: 11before loopopen ›Open: 22before modelopen ›Open: 33after modelopen ›Open: 44before toolopen ›Open: 55the toolopen ›Open: 66after toolopen ›Open: 77after loopopen ›Open: 88before replyopen ›It readsthe reply, and whether a hand-offreally existspromise gaterule · agentpassCarries onfailIf it failshand to a personWHERE IN THE CODE: promise.honest() · F promise.py:126
The strip at the top shows every check point where this check runs, in dark.

How we took care of it.

  1. If the reply promises "we'll call you" or "a colleague will help", a real escalation must be behind it.
  2. If not, raise one.
It readsthe reply, and whether a hand-off really exists
Kindrule
Provided byagent
If it failshand to a person
Where in the codepromise.honest() · F promise.py:126
Runs at7 · After the loop
Real exampleGap F-122: the safe reply itself promises a colleague without raising one (flow 12).

More on this: Hand-off flow.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

async def honest(
    result: TurnResult,
    desk: Handoff,
    conversation: Conversation,
    identity: Identity,
    run_id: RunId,
) -> TurnResult:
    """Make the promise true, or withdraw it.

    Called **after** Tier 2, because that is the first point at which the
    question is answerable: until the desk has had its chance, a turn that looks
    like an unbacked promise may be one about to be handed to a person.
    Afterwards `Completed` means what its type says — nothing pending, nothing
    open, nothing that will produce what was just promised.

    The desk is asked for a handoff, which is the continuation this agent has.
    Where there is none, **the words are replaced and the result is not** — the
    turn keeps its type and its termination, so a run stopped by the cost ceiling
    is still recorded as stopped by the cost ceiling. An earlier version returned
    the desk's refusal instead and lost that: the trace said `refused`, and why
    the agent gave up was gone. Removing a false promise must not also remove the
    record of what happened.

    A held turn does not pass here. Its replies are the desk's own constants, and
    every one of them names the ticket that backs it.
    """
    if not isinstance(result, Completed):
        return result
    promise = commits(result.reply)
    if not promise:
        return result
    tel.counters.promises.add(1, {"kind": promise})
    with tel.span(
        "agent.promise.unbacked",