cleandataengineering.aiEval Harness, drawn simply
You are here: Architecture › Inside the engine

Eval Harness Tutorial · Architecture

Inside the engine: ports and adapters

A port is a fixed shape: “give me spans”, “judge this request”, “store this result”. The engine only ever talks to ports. Each real tool sits behind an adapter that fits the port, chosen by name in YAML. That's why swapping Phoenix for App Insights, or Presidio for Azure Content Safety, never touches a rule or a report.

The engine's five steps are drawn on the whole picture; this page lists the ports those steps talk through.

PortSideWhat it doesState
TraceSourceUpstreamReads traces or datasets; yields spans to rebuild sessions fromBuilt OTLP file built
InlineHookHostLets a framework, gateway or CI step call our checks at a momentNot built yet Tier 7
EvaluatorCalledOne check: request in, result outBuilt 8 checks
JudgeCalledOne judge-model setting, handed to each SDK in its own shapeBuilt
RuleRunnerOursRuns rules over many records (SQL), reused from llm-cost-explainerNot built yet Tier 5
StateStoreOursRunning totals: budgets, rates, repeat callsNot built yet Tier 3
PolicyOursTurns results into one action: allow, redact, block, escalate, alertNot built yet Tier 3
ResultSinkDownstreamWrites results into the team's toolsNot built yet Tier 4
AlertSinkDownstreamSends alerts when a rule over results firesNot built yet Tier 8

How a check's adapter is built, and why it has three parts, is on Design.