cleandataengineering.aiAgent Harness, drawn simply
You are here: Tech stack

Agent Harness Tutorial · Tech stack

Tech stack: what we chose, and what else there was

For each of the 22 concerns: what it does, what we chose, what Azure offers for it, and what open source offers. They are in the order a turn meets them, then the ones around every turn, then the ones between releases.

Goes inA concern, for example "Model invocation"
What happensLay out the Azure options and the open-source options next to our choice, each marked used, planned, considered or dropped
Comes outA choice you can defend, and a known way out if it changes
Click a concern to see its three-column comparison.

The rule from the Series: the specs fix the concern and the port; Azure only fills the binding. The order of preference for each port is a managed Azure service, then open source run on Azure, then our own code, and our own code only where no product does the job. See what stays ours.

✓ used · ◐ planned or recommended · · considered, not picked · ✗ dropped

ConcernFamilyOur choiceAzure optionsOpen-source options
CH · Channels & human deskActOur chat page; Desk API /ops; ChatwootCommunication Services (◐); Bot Service (◐); Web PubSub (·); Dynamics 365 Contact Center (·)Chatwoot (◐); A Next.js window alone (·)
L16 · Identity & authzControlMicrosoft Entra ID; Managed identity + Key Vault; PyJWTEntra ID · managed identity · Key Vault (✓); Entra External ID (◐); azure-identity SDK (✗); MSAL (✗)PyJWT (✓); Keycloak (·); HashiCorp Vault (·)
PV · Privacy & data protectionControlredact() in telemetry; no_pii_echo rule; India regions; forget() + 30-day retentionAI Language (PII) (◐); Encryption at rest, private endpoints (◐)Presidio (·)
L7 · Policy enforcementControlOur positioned rules; Content Safety via APIM; APIM allow-list + rate limitAI Content Safety (✓); Prompt Shields on tool results (◐); APIM llm-content-safety policy (✗); Groundedness detection (◐)Llama Prompt Guard 2 (◐); Presidio + spaCy (·); LLM Guard (✗); Llama Guard (✗); NeMo Guardrails (✗); Guardrails AI (✗)
L8 · Concurrency & flowActAPIM rate limit; One turn per conversation; Service Bus queueService Bus Premium (◐); APIM rate limit (✓); APIM llm-token-limit (✗)Own semaphores (·); LiteLLM limits (·)
L5 · State & memoryThinkAzure PostgreSQL Flexible Server; psycopg + pool; 5 agent tables + DBOSAzure Database for PostgreSQL (✓); Cosmos DB (·); Managed Redis (·)PostgreSQL / Redis containers (·)
18 · Knowledge & retrievalThinkpgvector on the same PostgreSQLAzure AI Search (·); Content Understanding (·)pgvector (◐); Qdrant on Container Apps (·)
L1 · Context assemblyThinkOur assembler; Prompts in Git; jinja2Foundry prompt management (✗)Jinja templates (✓)
L2 · Model invocationThinkAPIM AI gateway; Groq gpt-oss-120b; Pydantic AIAPIM AI gateway (✓); Foundry Models (◐); Azure Model router (✗); APIM backend pools (◐)Pydantic AI (✓); LiteLLM proxy (·); vLLM on AKS (·)
L4 · Control loopThinkHand-written loop; BudgetsFoundry Agent Service (✗); Microsoft Agent Framework (✗)Pydantic AI Agent (✗); LangGraph (✗); Claude Agent SDK (✗); Anthropic tool runner (·)
L3 · Tool layerActMCP Python SDK; jsonschema; Our gating, keys, size capsMCP servers on Container Apps (✓); APIM in front of MCP (◐); Foundry tools (·)MCP Python SDK (✓); FastMCP (·)
17 · Agent-to-agentActA2A endpoints; API Center registryAPI Center (◐); Foundry connected agents (·); Agent Framework A2A (·)A2A SDK (◐); Backstage (·)
L14 · Human-in-the-loopControlDBOS 3.2.0; Desk API /ops; Teams cardsDurable Task Scheduler (✗); Durable Functions (✗); Logic Apps + Teams cards (✗)DBOS (✓); Temporal (·); Restate (·); Inngest (✗); Prefect (·)
L6 · I/O contractsActPydantic models; jsonschema; MCP structured outputAPIM schema validation (◐)Pydantic (✓); JSON Schema (✓)
L10 · Failure handlingControlResilientLLM; Typed failures; Ledger abandon-and-retryAPIM backend pools + breaker (◐); Container Apps health probes (◐)tenacity (·); Own resilience wrapper (✓)
L11 · ObservabilitySeeOpenTelemetry; azure-monitor-opentelemetry; alerts.bicepApplication Insights (✓); Managed Prometheus + Grafana (◐); Foundry tracing (·)OpenTelemetry (✓); Langfuse · Prometheus · Grafana (·); Jaeger (·)
L9 · Determinism & replaySeeOur run record + cassettes; Config fingerprint(nothing on Azure) (·)VCR.py (·); Own recorder (✓)
L13 · Cost accountingSeeOur meter + budgets; APIM token metric; $10/month budgetCost Management (✓); APIM token metrics (✓)Langfuse cost (·); Own meter (✓)
PS · Performance & scalabilityActStateless Container Apps; Per-turn time budget; Cache-friendly context orderContainer Apps autoscaling (✓); Azure Load Testing (◐); Managed Redis (·)(none named) (·)
AV · Availability & disaster recoveryControlStateless servers + durable state; Bicep + azdZone redundancy, PostgreSQL HA (◐); Front Door health probes (◐); Geo-redundant backups (◐)(none named) (·)
L12 · Eval harnessChangeAgentTwin; pytest + 4 CI gates; pydantic-evalsFoundry evaluation (◐); AI Red Teaming Agent (◐)AgentTwin · pytest · pydantic-evals (✓); promptfoo, PyRIT, garak (◐); DeepEval, Inspect (·)
L15 · Release & configChangeApp Configuration; Container Apps revisions; GitHub Actions + OIDCApp Configuration (✓); Container Apps revisions (✓); Container Registry (✗)OpenFeature + flagd, Unleash (·); Terraform, Argo CD (·)

Want every detail? The long version is in the Series: every service, and the binding cheat sheet.

Look inside

CH · Channels & human deskHow customers and staff reach the agent, and how a person takes over with the full history.L16 · Identity & authzWho is asking, and whose authority the agent acts with.PV · Privacy & data protectionPersonal data used only for this case, kept in India, masked, and forgotten on request.L7 · Policy enforcementGuardrails on the way in and on the way out.L8 · Concurrency & flowWho gets served first when load spikes.L5 · State & memoryWhat is kept, for how long, and for whom.18 · Knowledge & retrievalFinds sourced passages, filtered by what this user may see.L1 · Context assemblyWhat the model sees each turn, built in one place, within a budget.L2 · Model invocationEvery model call goes through one door.L4 · Control loopPlan, act, check, with step and no-progress limits.L3 · Tool layerActions through typed tools, checked before they run.17 · Agent-to-agentAgents hand work to agents through typed, bounded, traced hand-offs.L14 · Human-in-the-loopApprovals and hand-offs the agent cannot grant itself.L6 · I/O contractsTyped inputs and outputs at every edge.L10 · Failure handlingTyped failures, fallbacks, undo paths, honest messages.L11 · ObservabilityTraces, metrics and alerts for every step.L9 · Determinism & replayA past decision can be re-run and explained.L13 · Cost accountingSpend measured per unit of business work.PS · Performance & scalabilityFast enough at the 95th percentile; more servers when load rises; the model quota is the real limit.AV · Availability & disaster recoveryA lost server, zone or model goes unnoticed; a lost region is rebuilt from code and backups.L12 · Eval harnessGolden sets and graders that gate every release.L15 · Release & configPrompt, model and policy versioned and rolled back together.Every comparison, side by sidePlain Python vs Pydantic AI vs LangGraph vs Claude SDK vs Agent Framework; DBOS vs Temporal vs Durable Task Scheduler; and more.