Agent Harness Tutorial · Tech stack
Tech stack: what we chose, and what else there was
For each of the 22 concerns: what it does, what we chose, what Azure offers for it, and what open source offers. They are in the order a turn meets them, then the ones around every turn, then the ones between releases.
Goes inA concern, for example "Model invocation"
What happensLay out the Azure options and the open-source options next to our choice, each marked used, planned, considered or dropped
Comes outA choice you can defend, and a known way out if it changes
The rule from the Series: the specs fix the concern and the port; Azure only fills the binding. The order of preference for each port is a managed Azure service, then open source run on Azure, then our own code, and our own code only where no product does the job. See what stays ours.
✓ used · ◐ planned or recommended · · considered, not picked · ✗ dropped
| Concern | Family | Our choice | Azure options | Open-source options |
|---|---|---|---|---|
| CH · Channels & human desk | Act | Our chat page; Desk API /ops; Chatwoot | Communication Services (◐); Bot Service (◐); Web PubSub (·); Dynamics 365 Contact Center (·) | Chatwoot (◐); A Next.js window alone (·) |
| L16 · Identity & authz | Control | Microsoft Entra ID; Managed identity + Key Vault; PyJWT | Entra ID · managed identity · Key Vault (✓); Entra External ID (◐); azure-identity SDK (✗); MSAL (✗) | PyJWT (✓); Keycloak (·); HashiCorp Vault (·) |
| PV · Privacy & data protection | Control | redact() in telemetry; no_pii_echo rule; India regions; forget() + 30-day retention | AI Language (PII) (◐); Encryption at rest, private endpoints (◐) | Presidio (·) |
| L7 · Policy enforcement | Control | Our positioned rules; Content Safety via APIM; APIM allow-list + rate limit | AI Content Safety (✓); Prompt Shields on tool results (◐); APIM llm-content-safety policy (✗); Groundedness detection (◐) | Llama Prompt Guard 2 (◐); Presidio + spaCy (·); LLM Guard (✗); Llama Guard (✗); NeMo Guardrails (✗); Guardrails AI (✗) |
| L8 · Concurrency & flow | Act | APIM rate limit; One turn per conversation; Service Bus queue | Service Bus Premium (◐); APIM rate limit (✓); APIM llm-token-limit (✗) | Own semaphores (·); LiteLLM limits (·) |
| L5 · State & memory | Think | Azure PostgreSQL Flexible Server; psycopg + pool; 5 agent tables + DBOS | Azure Database for PostgreSQL (✓); Cosmos DB (·); Managed Redis (·) | PostgreSQL / Redis containers (·) |
| 18 · Knowledge & retrieval | Think | pgvector on the same PostgreSQL | Azure AI Search (·); Content Understanding (·) | pgvector (◐); Qdrant on Container Apps (·) |
| L1 · Context assembly | Think | Our assembler; Prompts in Git; jinja2 | Foundry prompt management (✗) | Jinja templates (✓) |
| L2 · Model invocation | Think | APIM AI gateway; Groq gpt-oss-120b; Pydantic AI | APIM AI gateway (✓); Foundry Models (◐); Azure Model router (✗); APIM backend pools (◐) | Pydantic AI (✓); LiteLLM proxy (·); vLLM on AKS (·) |
| L4 · Control loop | Think | Hand-written loop; Budgets | Foundry Agent Service (✗); Microsoft Agent Framework (✗) | Pydantic AI Agent (✗); LangGraph (✗); Claude Agent SDK (✗); Anthropic tool runner (·) |
| L3 · Tool layer | Act | MCP Python SDK; jsonschema; Our gating, keys, size caps | MCP servers on Container Apps (✓); APIM in front of MCP (◐); Foundry tools (·) | MCP Python SDK (✓); FastMCP (·) |
| 17 · Agent-to-agent | Act | A2A endpoints; API Center registry | API Center (◐); Foundry connected agents (·); Agent Framework A2A (·) | A2A SDK (◐); Backstage (·) |
| L14 · Human-in-the-loop | Control | DBOS 3.2.0; Desk API /ops; Teams cards | Durable Task Scheduler (✗); Durable Functions (✗); Logic Apps + Teams cards (✗) | DBOS (✓); Temporal (·); Restate (·); Inngest (✗); Prefect (·) |
| L6 · I/O contracts | Act | Pydantic models; jsonschema; MCP structured output | APIM schema validation (◐) | Pydantic (✓); JSON Schema (✓) |
| L10 · Failure handling | Control | ResilientLLM; Typed failures; Ledger abandon-and-retry | APIM backend pools + breaker (◐); Container Apps health probes (◐) | tenacity (·); Own resilience wrapper (✓) |
| L11 · Observability | See | OpenTelemetry; azure-monitor-opentelemetry; alerts.bicep | Application Insights (✓); Managed Prometheus + Grafana (◐); Foundry tracing (·) | OpenTelemetry (✓); Langfuse · Prometheus · Grafana (·); Jaeger (·) |
| L9 · Determinism & replay | See | Our run record + cassettes; Config fingerprint | (nothing on Azure) (·) | VCR.py (·); Own recorder (✓) |
| L13 · Cost accounting | See | Our meter + budgets; APIM token metric; $10/month budget | Cost Management (✓); APIM token metrics (✓) | Langfuse cost (·); Own meter (✓) |
| PS · Performance & scalability | Act | Stateless Container Apps; Per-turn time budget; Cache-friendly context order | Container Apps autoscaling (✓); Azure Load Testing (◐); Managed Redis (·) | (none named) (·) |
| AV · Availability & disaster recovery | Control | Stateless servers + durable state; Bicep + azd | Zone redundancy, PostgreSQL HA (◐); Front Door health probes (◐); Geo-redundant backups (◐) | (none named) (·) |
| L12 · Eval harness | Change | AgentTwin; pytest + 4 CI gates; pydantic-evals | Foundry evaluation (◐); AI Red Teaming Agent (◐) | AgentTwin · pytest · pydantic-evals (✓); promptfoo, PyRIT, garak (◐); DeepEval, Inspect (·) |
| L15 · Release & config | Change | App Configuration; Container Apps revisions; GitHub Actions + OIDC | App Configuration (✓); Container Apps revisions (✓); Container Registry (✗) | OpenFeature + flagd, Unleash (·); Terraform, Argo CD (·) |
Want every detail? The long version is in the Series: every service, and the binding cheat sheet.
Look inside
CH · Channels & human deskHow customers and staff reach the agent, and how a person takes over with the full history.L16 · Identity & authzWho is asking, and whose authority the agent acts with.PV · Privacy & data protectionPersonal data used only for this case, kept in India, masked, and forgotten on request.L7 · Policy enforcementGuardrails on the way in and on the way out.L8 · Concurrency & flowWho gets served first when load spikes.L5 · State & memoryWhat is kept, for how long, and for whom.18 · Knowledge & retrievalFinds sourced passages, filtered by what this user may see.L1 · Context assemblyWhat the model sees each turn, built in one place, within a budget.L2 · Model invocationEvery model call goes through one door.L4 · Control loopPlan, act, check, with step and no-progress limits.L3 · Tool layerActions through typed tools, checked before they run.17 · Agent-to-agentAgents hand work to agents through typed, bounded, traced hand-offs.L14 · Human-in-the-loopApprovals and hand-offs the agent cannot grant itself.L6 · I/O contractsTyped inputs and outputs at every edge.L10 · Failure handlingTyped failures, fallbacks, undo paths, honest messages.L11 · ObservabilityTraces, metrics and alerts for every step.L9 · Determinism & replayA past decision can be re-run and explained.L13 · Cost accountingSpend measured per unit of business work.PS · Performance & scalabilityFast enough at the 95th percentile; more servers when load rises; the model quota is the real limit.AV · Availability & disaster recoveryA lost server, zone or model goes unnoticed; a lost region is rebuilt from code and backups.L12 · Eval harnessGolden sets and graders that gate every release.L15 · Release & configPrompt, model and policy versioned and rolled back together.Every comparison, side by sidePlain Python vs Pydantic AI vs LangGraph vs Claude SDK vs Agent Framework; DBOS vs Temporal vs Durable Task Scheduler; and more.