Eval Harness Tutorial · Start here
The whole picture, one step at a time
Why the steps are separate. Each step meets the next through a fixed shape (a port). A Phoenix reader and an App Insights reader both produce the same Session, so the checks never know where the data came from. A Presidio check and one of our patterns both answer with the same result, so the reports never know who judged. Swapping a tool means adding one adapter and changing one line of YAML.
Why “decide” is its own step. A check only says what it found. Whether that blocks a reply, raises an alert or is just counted is a separate choice, so the same check can block in one company and only alert in another.
We never replace a team's tools. No database, dashboard or chat UI of our own: we read from where traces already are and write verdicts back there.
- Is the five-step picture clear enough to explain to an ops person in two minutes? If not, which step is unclear?
Decided: Yes. The five steps stay the explanation. - Is anything missing that an ops team would expect, such as a weekly report or alerts?
Decided: Yes: a weekly report and alerts. Both belong to step 5 (decide and record), now drawn there as not built yet.