cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 2 · Before the model › rate limit

Agent Harness Tutorial · End to end · Zoom level 3

How we take care of it: rate limit

rate limit catches more than 30 calls a minute.

Open: 11before loopopen ›Open: 22before modelopen ›Open: 33after modelopen ›Open: 44before toolopen ›Open: 55the toolopen ›Open: 66after toolopen ›Open: 77after loopopen ›Open: 88before replyopen ›It readscalls per minute on the agent'sAPIM subscriptionrate limitrule · Azure APIMpassCarries onfailIf it fails429, wait, retryWHERE IN THE CODE: infra/policies/groq-api.xml:35 · ResilientLLM
The strip at the top shows every check point where this check runs, in dark.

How we took care of it.

  1. APIM allows 30 calls a minute.
  2. Over that it answers 429; our layers call that Throttled, wait as told, and retry within the turn's 4 retries.
It readscalls per minute on the agent's APIM subscription
Kindrule
Provided byAzure APIM
If it fails429, wait, retry
Where in the codeinfra/policies/groq-api.xml:35 · ResilientLLM
Runs at2 · Before the model
Real exampleGroq's own free tier, 8,000 tokens a minute, is the tighter limit in practice: 3–5 loop turns a minute.

More on this: Layers around the call.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

    <!-- Request rate per subscription key (the agent's): 30 calls a minute. -->
    <rate-limit calls="30" renewal-period="60" remaining-calls-header-name="x-ratelimit-remaining" />