cleandataengineering.aiAgent Harness, drawn simply
You are here: Tech stack › All comparisons

Agent Harness Tutorial · Tech stack

Every comparison, side by side

Every technology choice in the harness, as a head-to-head table: the options, the same criteria for each, a verdict, and when we would switch. Tables from the Series say so; the others were added here and are marked "added".

Compared: Who owns the control loop?

Plain Python (…ourschosenPydantic AI Ag…open sourcemodel layer; A1–A3LangGraphopen sourceworkflow-shaped onlyClaude Agent S…vendorinternal agents onlyAnthropic tool…vendorsmall Claude agentsMicrosoft Agen…open source.NET teams onlyFoundry Agent …Azurerejected
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictWho owns the loopAny modelCan tests reach every stepLock-inObservabilityBest for
Plain Python (hand-written)ourschosenusyesyesnoneour OTel spansassured, transactional agents
Pydantic AI Agentopen sourcemodel layer; A1–A3framework; we can step ityesyes if we step the looplow: open source, any cloudOTel built intyped, testable Python agents
LangGraphopen sourceworkflow-shaped onlyframework runs our graphyesyes, node by nodemedium: graph model, LangSmithfull only with LangSmithworkflow agents with approvals
Claude Agent SDKvendorinternal agents onlythe SDKClaude onlypartly: the loop is hiddenhigh: Claude-onlypartial tracescoding, ops, research agents
Anthropic tool runner (thin helper)vendorsmall Claude agentsus, with hooksClaude onlyyes: the steps are oursmedium: Claude-onlyour OTel spanssmall Claude-only agents
Microsoft Agent Frameworkopen source.NET teams onlyframeworkyestrajectory checks re-provedmedium: framework modelOTel, Azure-native.NET or Azure-first teams
Foundry Agent Service (prompt agent)AzurerejectedAzureFoundry modelsno: loop and retries hiddenhigh: Azure-only serviceFoundry tracinglow-code agents, not assured ones
DecisionWe write the loop ourselves and use Pydantic AI only as the model layer, so budgets and checks sit at exact, testable positions. We would use LangGraph for workflow-shaped agents, Agent Framework for a .NET team, and full Pydantic AI for simple A1–A3 agents.

On its concern page: L4 · Control loop

Compared: What keeps a run alive while it waits days for a person?

DBOSopen sourcechosenTemporalopen sourcefor long sagasDurable Task…Azureonly if Azure-onlyAzure Durabl…Azureif already on FunctionsLogic Apps S…Azureback-office onlyRestateopen sourcewatchPrefectopen sourcedata and eval jobsDIY on Postg…ourssimplest waits only
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictWhere state livesNew infrastructureRules on our codeApprovals and long waitsOps burden for usExit cost
DBOSopen sourcechosenour Azure PostgreSQLnone: reuses our PostgreSQLlightdurable sleep, messages, eventslowestlow: open source, plain Postgres
Temporalopen sourcefor long sagasTemporal's store + visibility indexTemporal Cloud, or AKS clusterstrict: no clock, random, I/Osignals, updates, timers: richestlow on Cloud; high self-hostedlow in code; in-flight stays
Durable Task SchedulerAzureonly if Azure-onlymanaged by Azurea DTS resourcestrict, like Temporalexternal events, durable timerslowhigh: own model, Azure-only
Azure Durable FunctionsAzureif already on FunctionsAzure Storage, SQL or DTSrun the agent on Functionsstrict replay rulesexternal events, durable timerslow: serverlessmedium: tied to Functions
Logic Apps StandardAzureback-office onlymanaged by Azurea Logic Apps resourcenot code-first: designerbuilt-in approval emailslowhigh: designer workflows
Restateopen sourcewatchthe Restate server's storageRestate server or Clouddurable steps via its contextdurable calls, timersanother service to run, securemedium: younger ecosystem
Prefectopen sourcedata and eval jobsPrefect server databasePrefect server or Cloudlight: Python flows, tasksbuilt for batch, not peopleserver plus workersmedium
DIY on PostgreSQLourssimplest waits onlyour PostgreSQLnonenone: it is all oursstatus rows + scheduled jobwe own every bugnone
DecisionWe chose DBOS because its state lives in the PostgreSQL we already run, so it adds no infrastructure. We would move to Temporal for long undo flows (sagas) or mature versioning, and to Durable Task Scheduler only if the system must be Azure-native PaaS.

On its concern page: L14 · Human-in-the-loop

Compared: How do agents reach models, and who hosts them?

APIM AI gatewayAzurechosenLiteLLM proxyopen sourceexit pathPortkey gatewayvendornot evaluatedDirect SDK fro…open sourcerejectedGroqvendordev backendFoundry ModelsAzureplannedAzure OpenAIAzurenot default
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictRoleLimits and token metricsContent screeningFailoverLock-inStatus here
APIM AI gatewayAzurechosenone door for every model callrate limit, token metriccalls Content Safetybackend pools (planned)medium: policies are APIM configbuilt (Consumption tier)
LiteLLM proxyopen sourceexit pathOpenAI-style proxy, many providersbudgets, rate limits, spendvia guardrail hooksfallbacks between modelslow: open sourcethe gateway in the open stack
Portkey gatewayvendornot evaluatedgateway, open core plus SaaSlimits, cost logsvia guardrail plug-insfallbacks, retriesmedium: SaaS featuresnamed only
Direct SDK from each agentopen sourcerejectedeach agent calls the providernone in one placenone in one placeeach agent codes its ownprovider keys in every agentrejected
Groqvendordev backendhosts open-weight modelsGroq's own rate limitsnone: APIM adds itsingle backendlow: OpenAI-compatible APIbuilt: gpt-oss-120b
Foundry ModelsAzureplannedGPT, Claude, small modelsvia APIMbuilt-in content filtersvia APIM backend poolslow: one catalogue, many modelsrecommended (Phase 2)
Azure OpenAIAzurenot defaultGPT deployments on Azurevia APIMbuilt-in content filtersvia APIM backend poolshigh: re-tune prompts, evalsClaude in Foundry preferred
DecisionEvery model call goes through the APIM AI gateway, today to Groq and later to Foundry Models, so limits, metrics and screening live in one place. LiteLLM is the gateway we would use off Azure; business logic stays out of the gateway so that move stays cheap.

On its concern page: L2 · Model invocation

Compared: What decides the route, and which model answers?

Rules (regex rout…ourschosenLocal classifier …open sourceplannedSmall LLM (gpt-5-…vendorplannedModelSelector (go…oursplannedAzure Model routerAzurerejectedfastText / Azure …vendorrejected
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictDecidesWhenNetwork callSame input, same answerStatus
Rules (regex router)ourschosenroute: refuse, hand off, direct, loopevery turn, before any modelnoneyesruns today (layer 1)
Local classifier (Model2Vec + sklearn)open sourceplannedthe intent when rules are unsureevery message, about 1 msnone: a versioned fileyesTier 6; needs ~500 labels
Small LLM (gpt-5-nano labels)vendorplannedroute when classifier unsurerarelyone small callnot guaranteedTier 6
ModelSelector (golden set)oursplannedwhich model each job usesoffline, before each releaseoffline onlyyes: same cases, same verdictplanned: a CI job
Azure Model routerAzurerejectedwhich model answers a promptrun time, every callevery callnot guaranteednot used: model is Groq
fastText / Azure CLUvendorrejectedthe intentevery messageCLU: yesyesarchived; CLU retiring 2029
DecisionRules route every message today; a local classifier, then a small LLM only when unsure, are planned for Tier 6. Model choice is made offline by the ModelSelector on the golden set; Azure Model router is not used because its choice is not reproducible.

On its concern page: Router

Compared: Who provides each guardrail check?

Our rules (h…ourschosenAzure Conten…AzurechosenLlama Prompt…open sourceto buildPresidioopen sourceto decideLLM Guardopen sourcerejectedLlama Guardopen sourcerejectedNeMo Guardra…open sourcerejectedGuardrails AIopen sourcerejected
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictKindWhere it runsWhat it catchesWhy
Our rules (harness + agent)ourschosenrules in codeevery position, before to reply30 checks, all builtevery check that protects money
Azure Content Safety (via APIM)Azurechosenservicegateway: prompt and replyPrompt Shields, harm categoriesallow-list, rate limit, harms
Llama Prompt Guard 2open sourceto buildsmall classifier modelpre_model and tool resultsprompt injectionfast; covers tool results (F-132)
Presidioopen sourceto decideclassic ML (spaCy)before the modelpersonal data to maskthe model may need identifiers
LLM Guardopen sourcerejectedscanner librarynot placedinvisible text, secrets, topicsnormalised() and router cover it
Llama Guardopen sourcerejectedLLM safety classifiernot placedharmful contenttoo slow for every turn
NeMo Guardrailsopen sourcerejecteddialogue rails frameworknot placedtopic and dialogue railswants to own the loop
Guardrails AIopen sourcerejectedvalidator frameworknot placedoutput validatorsoverlaps LLM Guard
DecisionOur own positioned rules make every check that protects money, and Azure Content Safety at the gateway adds Prompt Shields and harm screening. Prompt Guard 2 is the next addition for tool results; LLM Guard's Language scanner comes in only if we add Hindi replies.

On its concern page: L7 · Policy enforcement

Compared: Which library checks and gets identity tokens?

PyJWT + PyJWKClie…open sourcechosenPlain httpx form …ourschosenAzure SDK (azure-…Azuredrop hereMSALvendorrejectedIdentity port (Ve…ourskeepKeycloak adapteropen sourcereference agent
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictWhat it doesChecks tokens users sendGets tokens for usHides failure kindsWhy
PyJWT + PyJWKClientopen sourcechosenchecks signature, issuer, audienceyes, locally, keys cachednonono Entra call per request
Plain httpx form postsourschosensign-in, refresh, on-behalf-ofnoyes: one form post eachno: refused, misconfigured, downcode branches on each failure
Azure SDK (azure-identity)Azuredrop heregets tokens to call Azurenoyes: managed identitypartlywrong tool for checking tokens
MSALvendorrejectedsign-in, refresh, OBO, cachingnoyes, with caching, retriesyes: hidden caching, retrieshides the failures we branch on
Identity port (Verifier, Exchange, Login)ourskeepone interface, many adaptersvia the adaptervia the adapternoleaving Entra is configuration
Keycloak adapteropen sourcereference agentlocal identity providervia the Verifier portvia the Login portnoruns the reference agent off Azure
DecisionWe check tokens locally with PyJWT and get tokens with plain httpx form posts, all behind our identity port. MSAL and azure-identity would hide the failure kinds our code branches on; we would only revisit if token flows grew beyond single form posts.

On its concern page: L16 · Identity & authz

Compared: How does the agent call tools?

MCP Python SDK (m…open sourcechosenFastMCPopen sourcealternativeFunction tools in…open sourcerejectedOpenAPI / REST to…open sourcerejectedAPIM REST-as-MCPAzurerejectedFoundry toolsAzureonly on Foundry
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictWhat it isProcess boundaryPer-tool metadataAuth on every callVerdict reason
MCP Python SDK (mcp)open sourcechosenofficial MCP client and serveryes: its own server_meta: side effect, scopebearer token checked againMCP is the only path
FastMCPopen sourcealternativehigher-level MCP server libraryyesyesyes, via its authlisted beside the official SDK
Function tools in processopen sourcerejectedPython functions given to modelno: same processonly what we addinherits the agent's rightsno far-end checks
OpenAPI / REST toolsopen sourcerejectedHTTP API described by OpenAPIyesno standard placeyesnot shaped by intent
APIM REST-as-MCPAzurerejectedAPIM exposes REST as MCPyesnoat the gatewaywraps REST, not intent
Foundry toolsAzureonly on Foundrytools in Foundry Agent ServiceFoundry runs themFoundry's ownFoundry connectionsonly if the loop moves there
DecisionTools are dedicated MCP servers built with the official MCP Python SDK, each tool declaring its side effect and required scope, and the far end checks the token again. We would put APIM in front as the MCP front door, but never use REST-as-MCP for agent-facing tools.

On its concern page: L3 · Tool layer

Compared: Where does conversation state live?

Azure PostgreSQL Flex…AzurechosenCosmos DBAzurefor global scaleAzure Managed RedisAzureoptionalSQLite / file storeopen sourcelocal onlyIn-memory storeourstests only
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictHolds hereOne transaction with ledgerFits DBOS and pgvectorHA on AzureWhy
Azure PostgreSQL Flexible ServerAzurechosenstate, ledger, approvals, findingsyesyeszone-redundant HAone store, Entra login, backups
Cosmos DBAzurefor global scalenothing todaywithin one partition onlynomulti-region built inonly for global scale
Azure Managed RedisAzureoptionalcounters and short caches onlynonozone redundancy availablefast, but not the record
SQLite / file storeopen sourcelocal onlyreference agent, on a laptopyes, one processnononesimple local runs
In-memory storeourstests onlytest staten/anononefast, nothing survives a restart
DecisionAll durable state lives in Azure PostgreSQL, because the ledger, outbox and checkpoint commit together and DBOS and pgvector reuse the same database. Cosmos DB would be considered only for global scale, and Redis only for counters and short caches.

On its concern page: L5 · State & memory

Compared: Where do knowledge passages live and get searched?

pgvector in Azure PostgreS…open sourcechosenAzure AI SearchAzurelaterQdrant on Container Appsopen sourcealternativePineconevendornot evaluated
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictWhere it runsHybrid keyword + vectorEntitlement filterExtra infrastructureWhen to pick
pgvector in Azure PostgreSQLopen sourcechoseninside our PostgreSQLyes, with Postgres full texta SQL filter before rankingnone: one store, one backupdefault
Azure AI SearchAzurelatermanaged search serviceyes, plus semantic rankerfilters on indexed fieldsa search servicecorpus outgrows Postgres
Qdrant on Container Appsopen sourcealternativea container we runyes, dense plus sparsepayload filtersa service plus its storagededicated vector store needed
Pineconevendornot evaluatedmanaged SaaSyes, sparse-densemetadata filtersa SaaS outside Azurenamed only
DecisionPassages live in pgvector in the same PostgreSQL, so entitlement is a SQL filter applied before ranking and there is one store to back up. We would switch to Azure AI Search when the corpus outgrows Postgres or we need its semantic ranker.

On its concern page: 18 · Knowledge & retrieval

Compared: Where do traces, metrics and alerts go?

Application In…AzurechosenManaged Promet…AzureplannedLangfuseopen sourcereference agentPrometheus + G…open sourcereference agentFoundry tracin…Azurewith FoundryOTel Collector…open sourcealternativeLangSmithvendornot used
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictHoldsWhere usedCustomer's wordsAlertsStatus
Application Insights (OTel distro)Azurechosenspans: one story per conversationAzurenot captured (privacy)log alerts, budget alerton Azure today
Managed Prometheus + GrafanaAzureplannedcounters, alert rules, dashboardsAzureneversame rules as referenceplanned, Tier 12
Langfuseopen sourcereference agenttraces and scoresreference agent, locallyon sampled turns, redactedvia the watch jobsecond exporter option
Prometheus + Grafana (self-run)open sourcereference agentcounters, 21 alerts, dashboardreference agent, locallyneverAlertmanagerDocker compose
Foundry tracing viewAzurewith Foundryagent traces in FoundryAzurenot decidednocomes with a Foundry model
OTel Collector + Jaegeropen sourcealternativespans, routed anywhereanywhereas configurednolisted open source option
LangSmithvendornot usedtraces, datasets (paid)with LangGraphas configuredyesnot used
DecisionSpans go to Application Insights and counters to Managed Prometheus, both fed by one OpenTelemetry setup. Langfuse plays the trace-store role in the reference agent and could be added as a second exporter; swapping is one adapter and one YAML line.

On its concern page: L11 · Observability

Compared: Which tools run and judge our evaluations?

pydantic-eva…open sourcechosenDeepEvalopen sourcealternativeRagasopen sourcealternativepromptfooopen sourceprompt-only jobsInspectopen sourcealternativeAzure AI Fou…AzureplannedPyRIT (AI Re…open sourceto adoptgarakopen sourceto adopt
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictJob hereDrives our agent loopNeeds a judge modelWhen it runsStatus
pydantic-evalsopen sourcechosenoffline eval runneryes, through our harnessfor soft graders onlypull request, releaseused
DeepEvalopen sourcealternativesoft graders, case draftingwith an adapteryes, for LLM metricsreleasealternative runner
Ragasopen sourcealternativeretrieval scores, case draftingnoyes, for most metricsreleasealternative to Foundry RAG
promptfooopen sourceprompt-only jobsmodel × test matrix; red teamno: needs an adapterfor model-graded checkspull requestfor prompt-only jobs
Inspectopen sourcealternativeeval framework, soft graderswith an adapterfor model-graded scorersreleasealternative runner
Azure AI Foundry evaluationAzureplannedgeneric LLM judgesno: judges our recordsyes, or Azure-billedrelease, sampled onlineplanned, Tier 12
PyRIT (AI Red Teaming Agent)open sourceto adoptbroad attack libraryvia our oraclesyes, attacker modelweekly, before releaseto adopt
garakopen sourceto adoptLLM vulnerability scannerno: probes the modelno for most probesweekly, before releaseto adopt
DecisionAgentTwin plus pytest drive the agent in a simulated world, with pydantic-evals as the runner and our own hard-check graders; Foundry evaluators supply generic judges once calibrated. promptfoo stays for prompt-only jobs, and PyRIT and garak are added for weekly red-team runs.

On its concern page: L12 · Eval harness

Compared: How do we replay a run without live calls?

Own cassettes at the model…ourschosenVCR.py (vcrpy)open sourceto adoptPydantic AI TestModel / Fu…open sourcechosenLangfuse / LangSmith replayvendornot used
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictWhat it recordsLevelOn a missStatus
Own cassettes at the model choke pointourschosenrequest fingerprint → responsemodel callsrun fails, never goes livebuilt; used in release gates
VCR.py (vcrpy)open sourceto adoptHTTP requests and responsesHTTP, for tool callsconfigurable record modeto adopt for tools
Pydantic AI TestModel / FunctionModelopen sourcechosennothing: scripted repliesmodel layern/aused in tests
Langfuse / LangSmith replayvendornot usedtraced promptsone model call, in a UIn/a: re-runs livenot used
DecisionWe record model exchanges ourselves at the one model choke point and refuse to replay under a different model, tool list or temperature, so a miss fails loudly. VCR.py is to be adopted for HTTP-level tool cassettes; trace-tool playgrounds re-run live, so they are not replay.

On its concern page: L9 · Determinism & replay

Compared: How is cost measured and capped?

Own Meter + cost fram…ourschosenAPIM token metricsAzurechosenLangfuse cost trackingopen sourcealternativeHeliconevendornot evaluatedAzure Cost ManagementAzurechosen
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictGrainStops a turnSourceStatus
Own Meter + cost frameworkourschosenper step, turn, successful taskyes: $0.50 per turnprice table × tokensbuilt (framework v0)
APIM token metricsAzurechosenper API and subscriptionno: Consumption lacks itgateway token countsbuilt
Langfuse cost trackingopen sourcealternativeper trace and generationnoits model price tablelisted open source option
Heliconevendornot evaluatedper request via its proxyrate limits onlyits price tablenot evaluated
Azure Cost ManagementAzurechosenper resource, per dayno: alerts onlythe Azure billbuilt: $10 budget alert
DecisionOur Meter caps every turn and computes cost per successful task, while APIM token metrics and Cost Management give the gateway and bill views. Langfuse or Helicone would only duplicate what the Meter and spans already record.

On its concern page: L13 · Cost accounting

Compared: What retries and fails over when a call breaks?

ResilientLLM (own…ourschosentenacityopen sourcealternativeProvider SDK retr…vendorrejectedPydantic AI fallb…open sourcechosenAPIM backend pool…AzureplannedHealth probes + F…Azurechosen
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictWhere it actsKnows our typed failuresShares one retry budgetStatus
ResilientLLM (own wrapper)ourschoseninside the LLMClient portyesyesbuilt
tenacityopen sourcealternativeany Python callonly what we tell itif we wire itlisted open source option
Provider SDK retriesvendorrejectedinside the openai SDKnono: invisible to usoff: max_retries=0
Pydantic AI fallback modelopen sourcechoseninside the model adapterpartlyyes, one budgetdesign: degradation path
APIM backend pools + breakerAzureplannedat the gatewaynomust share our budgetplanned, Tier 10
Health probes + Front DoorAzurechosencontainers and regionsnon/afinal (design)
DecisionModel retries live in our ResilientLLM wrapper with one retry budget, so SDK retries are switched off to stay visible. APIM backend pools and the circuit breaker are planned for production failover across backends and regions.

On its concern page: L10 · Failure handling

Compared: How are inputs, outputs and records typed?

Pydantic (frozen …open sourcechosenjsonschemaopen sourcechosendataclasses + jso…open sourcenot chosenattrs (+ cattrs)open sourcenot evaluatedmsgspecopen sourcenot evaluatedAPIM schema valid…Azureat the edge
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictValidationJSON Schema for toolsImmutabilityStatus
Pydantic (frozen models)open sourcechosenfull, with clear errorsyes: Field descriptionsyes: frozen=Truebuilt: every contract
jsonschemaopen sourcechosenvalidates against a schemauses the tool's scheman/abuilt: tool arguments
dataclasses + jsonschemaopen sourcenot chosennone built in; via jsonschemaby handyes: frozen=Truenot chosen
attrs (+ cattrs)open sourcenot evaluatedvalidators you writeno built-inyes: frozennot evaluated
msgspecopen sourcenot evaluatedtyped decoding, very fastyesyes: frozen Structsnot evaluated
APIM schema validationAzureat the edgeat the gatewayuses API scheman/adesign: edge check
DecisionEvery contract is a frozen Pydantic model, which also generates the tool schemas the model sees, and jsonschema validates tool arguments before any call. msgspec would be worth a look only if decoding speed ever mattered.

On its concern page: L6 · I/O contracts

Compared: What queues turns when load spikes?

Azure Service Bus Pre…AzurechosenPostgreSQL as queue (…ourschosenRedis Streamsopen sourcenot evaluatedAzure Storage QueuesAzurenot evaluatedAPIM rate limitsAzurechosen
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictOrdering and priorityDead-letterScales workers (KEDA)Extra infrastructureStatus
Azure Service Bus PremiumAzurechosensessions; priority via queuesbuilt inyesa namespacefinal (design)
PostgreSQL as queue (turns table)ourschosenleases; one turn per chatours: failed statusyes, query-basednoneturn status, to build
Redis Streamsopen sourcenot evaluatedper stream, consumer groupsours: pending listyesRedis we must keep durableRedis is for counters only
Azure Storage QueuesAzurenot evaluatedno ordering guaranteeno built-in dead-letteryesa storage accounttoo plain for deadlines
APIM rate limitsAzurechosennot a queue: 429 on overloadn/anonone: already therebuilt: 30 calls a minute
DecisionThe design queues turns on Service Bus Premium for dead-lettering and worker scaling, while turn status, leases and the outbox live in PostgreSQL so a reply exists only if the turn committed. APIM rate limits sit in front as the last line of flow control.

On its concern page: L8 · Concurrency & flow

Compared: How do config and releases change safely?

Azure App Conf…AzurechosenOpenFeature + …open sourcealternativeUnleashopen sourcealternativeContainer Apps…AzurechosenBicep + azdAzurechosenTerraformopen sourcealternativeArgo CDopen sourcenot used
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictJobAzure-nativePortable off AzureOps burdenStatus
Azure App ConfigurationAzurechosenflags, model per job, thresholdsyesnolow: managedbuilt: agent.enabled, limits
OpenFeature + flagdopen sourcealternativestandard flag API + flag daemonnoyeswe run flagdlisted open source option
Unleashopen sourcealternativefeature-flag server and UInoyeswe run a server, or SaaSlisted open source option
Container Apps revisionsAzurechosencanary traffic split, rollbackyesnolowfinal
Bicep + azdAzurechoseninfrastructure as codeyesno: Azure onlylowbuilt
Terraformopen sourcealternativeinfrastructure as codevia azurerm provideryes: many cloudsstate file to manage"Bicep or Terraform"
Argo CDopen sourcenot usedGitOps deploys to Kubernetesnoyesneeds Kubernetes (AKS)listed open source option
DecisionFlags, thresholds and model per job live in App Configuration, releases roll out as Container Apps revisions, and infrastructure is Bicep with azd. Terraform is the swap if the team already uses it or needs more than one cloud.

On its concern page: L15 · Release & config

Compared: How do agents hand work to other agents?

A2A SDK endpoints on …open sourcechosenFoundry connected age…Azureonly on FoundryMicrosoft Agent Frame…open source.NET teams onlyIn-process hand-off (…oursinside one agentAzure API Center (reg…Azurechosen
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictWhere agents runOpen protocolWorks across vendorsTyped hand-off and limitsStatus
A2A SDK endpoints on Container Appsopen sourcechosenour Container Appsyes: A2Ayesours: ticket, depth, fan-outfinal (design)
Foundry connected agentsAzureonly on FoundryFoundry Agent Serviceno: Foundry's ownFoundry agents onlyFoundry'sif the loop moves to Foundry
Microsoft Agent Framework A2Aopen source.NET teams onlyanywhereyes: A2A supportyespartly framework, partly ourslisted Azure option
In-process hand-off (one service)oursinside one agentone processnonooursfine within one agent
Azure API Center (registry)Azurechosenregistry, not a runtimelists A2A endpointsyesnofinal: who exists, which version
DecisionAgents hand off over the open A2A protocol on Container Apps, with our typed hand-off ticket, depth and fan-out limits and one trace, and are listed in API Center. Foundry connected agents would make sense only if the loop moved into Foundry.

On its concern page: 17 · Agent-to-agent

Compared: Which channels and which human desk?

Next.js window + …open sourcechosenNext.js window on…oursrejectedChatwoot widget +…open sourcerejected for customersAzure Communicati…AzurechosenAzure Bot ServiceAzureTeams onlyDynamics 365 Cont…vendorwhere it exists
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictCustomer windowStaff desk and hand-offChannelsVerdict reason
Next.js window + Chatwoot deskopen sourcechosenbest: our cards, upload, brandgood: queues, hand-offweb, WhatsApp, SMS, emailour default
Next.js window onlyoursrejectedbestnone: build itweb onlya contact centre isn't our job
Chatwoot widget + deskopen sourcerejected for customersgenericgoodweb, WhatsApp, SMS, emailclaims needs rich cards, upload
Azure Communication ServicesAzurechosenno windowno deskWhatsApp, SMS, emailthe channel pipes
Azure Bot ServiceAzureTeams onlynono human deskTeams connectornever as the desk
Dynamics 365 Contact Centervendorwhere it existsits ownbest: staff already use itmanyonly if already owned
DecisionCustomers use our own Next.js chat window, staff use the Chatwoot desk, and Communication Services carries WhatsApp, SMS and email. Where the insurer already runs a contact centre such as Dynamics 365, we hand off into that instead of Chatwoot.

On its concern page: CH · Channels & human desk

Compared: How is personal data found and masked?

Regex redact() (ours)ourschosenno_pii_echo reply ruleourschosenMicrosoft Presidioopen sourceto decideAzure AI Language PIIAzureplanned
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictWhere it runsIndia IDs (Aadhaar, PAN)Network callHandles free textStatus
Regex redact() (ours)ourschosenevery payload and log lineyes, Aadhaar Verhoeff-checkednoneknown patterns onlybuilt
no_pii_echo reply ruleourschosenpost_model and replyyesnoneknown patterns onlybuilt
Microsoft Presidioopen sourceto decidein our process (spaCy)built-in IN_PAN, IN_AADHAARnoneyes: names, placesto decide: model needs IDs
Azure AI Language PIIAzureplannedAzure serviceyes, India entity typesyesyesdesign: mask before model
DecisionOur regex redact() and the no_pii_echo rule are built and cover the identifiers that matter, on every log line and every reply. Masking before the model, with Presidio or Azure AI Language, is still to decide because the model sometimes needs the identifiers.

On its concern page: PV · Privacy & data protection