CCA-P start here · Mind maps
CCA-P mind maps
All 38 guide lines on one page. Each map says, for the situations the exam likes, which answer wins, including the row where the runner-up wins. Read these the week before the exam.
Domain 1: Solution Design & Architecture 17%
1.1 · Translate business problems into Claude-based AI solutions
Rule: If a part of the problem is exact (a lookup, a rule, a sum), give it to code; if it is language or judgement, give it to Claude, and stop at the simplest design that meets every stated constraint.
| When the stem says… | Pick | Why |
|---|---|---|
| Part of the task is an exact lookup or calculation | Plain code, a query or a tool | Exact, cheap and testable, with no model guesswork |
| The task is reading messy text and making a judgement | A Claude call with a clear prompt | This is where a language model adds value |
| The answer depends on private or changing documents | Claude plus retrieval (an augmented LLM) | The model needs the facts at request time |
| The decision is high-impact or regulated | Claude drafts, a person approves | Keeps accountability with a person |
| The steps cannot be known in advance | An agent | Only then does the extra autonomy pay for itself |
1.2 · Design end-to-end architectures (input → processing → output → feedback loops)
Rule: If quality drifts unseen, add the feedback loop; if bad data gets in, validate input; if bad formats get out, validate output in code.
| When the stem says… | Pick | Why |
|---|---|---|
| Quality drifted after launch and nobody noticed | Add the feedback loop and monitoring | Nothing was measuring quality in production |
| Blank or unsupported files cause invented answers | Validate at the input stage | Cheaper to reject early than to repair later |
| A downstream system breaks on bad formats | Validate output against a schema in code | Catches errors before delivery |
| Users fix outputs by hand every day | Capture the corrections as labelled data | Turns daily work into eval cases |
| One known case fails because the prompt is plainly wrong | Fix the prompt, then add the case to the evals | A direct defect is fixed directly; the loop stops it coming back |
1.3 · Select appropriate architectural patterns (workflow, agentic, augmented LLM)
Rule: Pick the least autonomy that solves the problem: one answer from documents gets an augmented LLM, fixed steps get a workflow, and open-ended work gets an agent.
| When the stem says… | Pick | Why |
|---|---|---|
| One question, answered from documents | Augmented LLM with retrieval | There is no sequence of steps to manage |
| Every item goes through the same known steps | Workflow: prompt chaining | A fixed order is predictable and testable |
| Inputs fall into distinct types that need different handling | Workflow: routing | Each type gets its own tuned prompt or model |
| A draft must meet clear criteria, and critique helps | Workflow: evaluator-optimizer | Repeated refinement against a checkable standard |
| The next step depends on what was just found | Agent | Only a model-directed loop can change the path as it goes |
1.4 · Design multi-agent systems and orchestration strategies
Rule: If the work splits into independent or specialist parts, use a coordinator with scoped subagents and full briefs; if the steps depend on each other, keep one agent with good tools.
| When the stem says… | Pick | Why |
|---|---|---|
| One user goal, a handful of tools, steps that depend on each other | A single agent | Coordination would add cost with no gain |
| Many independent research threads | Coordinator with parallel subagents | Breadth runs in parallel, in separate contexts |
| One part needs permissions the others must not have | A separate specialist subagent | Isolates tools and access by role |
| Subagents return inconsistent or repetitive results | Fuller briefs plus a coordinator check | Subagents know only what they are told |
1.5 · Apply decomposition techniques for complex problem solving
Rule: If the steps are known, chain focused prompts with a check between each; if one input is too big to review well, do per-item passes and then an integration pass; if the steps depend on findings, decompose adaptively.
| When the stem says… | Pick | Why |
|---|---|---|
| One prompt does extraction, analysis and writing, and quality is uneven | A prompt chain with checks between steps | Each step gets full attention and can be tested |
| Reviewing a forty-file change misses issues | Per-file passes, then an integration pass | Depth on each file plus a cross-file view |
| The plan depends on what the first findings show | Adaptive decomposition | A fixed chain cannot follow new leads |
| A simple task is already done well by one prompt | Keep the single prompt | Extra steps would only add latency and cost |
1.6 · Align solutions to business value pillars (efficiency, transformation, productivity, cost, performance SLAs)
Rule: If the sponsor wants lower cost, pick the cheapest design that meets the quality floor; if they want an SLA, design for the latency target; if they want something new, judge it by adoption; and always measure that pillar against a baseline.
| When the stem says… | Pick | Why |
|---|---|---|
| The goal is a lower cost per ticket | The cheapest configuration that meets the quality floor | Serves the cost pillar directly |
| Answers must arrive within an agreed time | A fast model and streaming on the live path | Meets the performance SLA |
| The goal is a service customers could not get before | A more capable design, judged by adoption and revenue | Transformation can justify a higher cost |
| Analysts want to handle more cases each day | Assisted drafting, with a person finishing | Productivity: more output per person |
| The sponsor's stated priority is the highest possible accuracy | The most accurate option, even at higher cost | Here the expensive option serves the right pillar |
Domain 2: Claude Models, Prompting & Context Engineering 13%
2.1 · Select appropriate Claude models based on trade-offs
Rule: If a smaller model passes your eval, use it; if only some requests are hard, route just those to a capable model; if nobody is waiting for the answer, batch it.
| When the stem says… | Pick | Why |
|---|---|---|
| Millions of simple, well-defined items a day | A smaller, faster model validated on an eval | Meets quality at a fraction of the cost |
| Hard multi-step reasoning or complex agent work | The most capable model | Smaller models fail the quality bar here |
| Traffic is a mix of easy and hard requests | Route by difficulty | Cheap for most requests, capable where needed |
| Results are needed by tomorrow, not instantly | Batch processing | Lower price, and latency does not matter |
| The small model fails the eval on important edge cases | Move up a tier, or route only those cases up | The quality bar comes before cost |
2.2 · Design system prompts, templates, and guardrails
Rule: If a rule is about style or scope, put it in the system prompt with its reason; if it must never be broken, also enforce it in code; and if many prompts share it, keep it in one versioned template.
| When the stem says… | Pick | Why |
|---|---|---|
| Instructions hidden in uploaded content change behaviour | Tag untrusted content as data, and check actions in code | Two layers against prompt injection |
| Teams copy and edit the same prompt, and versions drift | One versioned template with variables | A single reviewed source gives consistent behaviour |
| Output must always be valid JSON for a parser | Validate against a schema in code | A request in the prompt can still fail |
| The assistant drifts into off-topic advice | Clear scope and a hand-off path in the system prompt | Soft scope rules are well handled in the prompt |
| A rule must never be broken, such as a refund limit | Enforce it in the tool, not just the prompt | Code is deterministic; prompts are not |
2.3 · Apply prompt engineering techniques (zero-shot, few-shot, chain-of-thought)
Rule: If the task is clear and passes, stay zero-shot; if format or borderline cases wobble, add varied examples; if the format is right but the logic is wrong, let Claude reason before it answers.
| When the stem says… | Pick | Why |
|---|---|---|
| A clear, simple task that already passes | Zero-shot | Any extra technique adds cost without benefit |
| Output format or tone varies between runs | Few-shot examples | Examples show the exact target |
| Borderline cases are labelled inconsistently | Few-shot examples of those borderline cases | Examples show where the line falls |
| The format is right but multi-step decisions are wrong | Chain-of-thought or extended thinking | The failure is reasoning, not format |
| A quick lookup with a tight latency budget | No chain-of-thought | Reasoning would add delay for no gain |
2.4 · Optimize context windows and manage token usage
Rule: If context grows with history, summarise old turns and pin exact facts; if tools return bulk, trim it at the source; if documents are large, retrieve the relevant parts, and put any long document first and the question last.
| When the stem says… | Pick | Why |
|---|---|---|
| Cost rises with conversation length because the full history is resent | Summarise old turns and pin key facts | Smaller context, with exact values kept |
| A tool returns 200 fields and the task uses five | Trim tool output to the needed fields | Less noise and fewer tokens on every call |
| The whole policy library is pasted into every prompt | Retrieve the relevant sections | Sends only what the question needs |
| A short, stable reference text is needed on every request | Keep it in context and cache it | Retrieval would add a step for no saving |
| Answers miss a detail in a long document | Document first, question last | Helps the model use long inputs |
2.5 · Implement prompt reuse strategies (caching, modular prompts, Skills)
Rule: If the same long content goes with every request, put it first and cache it; if many prompts share rules, build them from modules; if expertise is needed only sometimes, package it as a Skill.
| When the stem says… | Pick | Why |
|---|---|---|
| The same long system prompt and policy go with every request; cost and latency matter | Static content first, then prompt caching | The repeated prefix is read from the cache |
| Many prompt variants share rules that keep drifting apart | Modular, versioned prompt components | One fix updates every prompt |
| Many specialist procedures, each needed only now and then | Agent Skills | Loaded on demand, not on every request |
| Each request is one-off, or its start changes every time | No caching | Nothing repeats, so the cache would not be reused |
| Caching is on but the hit rate is near zero | Move variable content after the static prefix | Something early in the prompt is changing |
Domain 3: Integration 19%
3.1 · Evaluate tool/agent configuration for capability bloat
Rule: If the role never uses a capability, remove it from both the tool list and the account; if it is rarely used but high-impact, keep it behind human approval.
| When the stem says… | Pick | Why |
|---|---|---|
| The role never uses a tool the agent has | Remove the tool | Eliminates the risk entirely |
| The role needs the tool, but rarely, and the action is high-impact | Keep it behind human approval | Removing it would block real work |
| The role needs a tool, but only part of it | Replace with a narrower tool | Keeps the job possible with less power |
| The agent runs under a shared or admin account | Give it its own scoped identity | The permission layer is the backstop if a tool slips through |
| Auditors need a record of what happened | Add logging as well | Logging supports, it never replaces, least privilege |
3.2 · Analyze authentication and authorization requirements to identify security gaps
Rule: If users can reach data they could not open themselves, act with their delegated, scoped token and filter retrieval by their access; if users are not signed in at all, fix authentication first.
| When the stem says… | Pick | Why |
|---|---|---|
| The agent acts for individual users in systems that already have per-user permissions | Delegated, user-scoped token | The system enforces what that user may do |
| Users see answers from documents they could not open themselves | Filter retrieval by each chunk's access list | Restricted text never reaches the model |
| Access rules are coarse and match teams cleanly (say, one index per legal entity) | Separate indexes per group | Simpler, and enough when permissions follow the split |
| A background job with no user behind it (nightly batch) | Its own narrowly scoped service identity | There is no user to delegate from, so least privilege applies to the service |
| Users are not signed in at all | Fix authentication first | Authorization needs a known identity to work from |
3.3 · Evaluate accuracy-latency trade-offs and justify configuration decisions
Rule: If the SLA is tight and only some requests are hard, route those to the heavy path; if a check's result is used later, move it off the live path.
| When the stem says… | Pick | Why |
|---|---|---|
| Most requests are easy, a few are hard, and the SLA is tight | Route: fast path by default, heavy path for hard cases | Keeps p95 low and accuracy where it matters |
| A check improves accuracy but the result is used later | Verify asynchronously, off the live path | Accuracy gain without user-facing delay |
| A person reads a long answer and complains it feels slow | Stream the output | Perceived speed improves; nothing else changes |
| Every request is high-stakes and the SLA is loose | The heavier, more accurate configuration everywhere | Latency has room, so take the accuracy |
| A long, identical prefix is sent on every call | Prompt caching | Lower time to first token at no accuracy cost |
3.4 · Analyze observability challenges and select monitoring strategies at scale
Rule: If you can't tell which step failed, trace every request under one ID; if spend or slowness hides in totals, measure per request and per step with percentiles; if quality may drift, score a live sample.
| When the stem says… | Pick | Why |
|---|---|---|
| Can't tell which step in a multi-step system caused a bad answer | End-to-end tracing with one request ID | Shows each step's input and output in context |
| Spend rose but volume didn't, or some users are very slow | Per-request, per-step token, cost and latency percentiles | Finds the expensive or slow path that totals and averages hide |
| Quality may be drifting, but nothing errors | Sampled quality evals with alerts | Quality is invisible to system metrics |
| The failing step is already known and needs detail | Richer logging on that step | Targeted detail is cheaper than logging everything |
3.5 · Design a RAG pipeline with appropriate chunking and indexing strategies
Rule: If ideas are split across chunks, chunk by the document's structure with overlap; if chunks lose their context, add metadata and contextual notes; if sources change, re-index.
| When the stem says… | Pick | Why |
|---|---|---|
| A clause or table is split across two chunks | Structure-aware chunking with overlap | Keeps each idea in one retrievable piece |
| Chunks are correct text but from the wrong version, region or date | Metadata plus filters, or contextual retrieval | The chunk alone doesn't say where it belongs |
| Short, unstructured documents such as chat messages or notes | Simple fixed-size or whole-document chunks | Little structure to exploit; simplicity wins |
| Answers became wrong right after a document refresh | Check the re-index and embedding model | Stale or mismatched index, not the model |
| Matches are precise but answers lack surrounding detail | Retrieve small chunks, pass the parent section | Precision for search, context for the answer |
3.6 · Apply retrieval strategies matched to data shape and query pattern
Rule: If queries mix exact codes with descriptions, use hybrid search; if the right result ranks too low, rerank; if the question needs totals over tables, give Claude a query tool.
| When the stem says… | Pick | Why |
|---|---|---|
| Users describe problems in their own words | Vector (semantic) search | Matches meaning across different wording |
| Queries contain exact codes, IDs or names, mixed with descriptions | Hybrid: vector plus BM25 keyword | Exact terms and meaning both count |
| Queries contain only exact identifiers, such as a part-number lookup | Keyword search or a direct lookup | No meaning to match; exact is enough |
| The right passage is retrieved but ranked too low | Add a reranker | Re-scores candidates for precision |
| Questions need totals, counts or trends over tables | A query tool over the database | Exact computation over every row |
3.7 · Evaluate connection protocols and select the appropriate integration mechanism (MCP, API/CLI, agent-to-agent)
Rule: If many AI clients need the same capability, build an MCP server; if one app makes a fixed call, call the API directly; if another party's agent runs the whole task, delegate agent-to-agent.
| When the stem says… | Pick | Why |
|---|---|---|
| The same tool is needed by several AI clients or teams | An MCP server | Build once, reuse everywhere, govern centrally |
| One backend calls a known service in a fixed way | Direct API call | Fewest moving parts; nothing to reuse |
| A coding agent with a shell needs a well-known tool | The existing CLI | The model already knows it; no server needed |
| A partner's agent handles a multi-step task with its own private systems | Agent-to-agent | You hand over a goal, not individual calls |
| A partner offers simple, documented endpoints and no agent | Direct API or an MCP wrapper | There is no agent to delegate to |
3.8 · Evaluate progressive discovery vs. monolithic context strategy
Rule: If content is large and each task uses a slice, discover it progressively with Skills, tool search or resources; if it is small, stable and always needed, load it up front and cache it.
| When the stem says… | Pick | Why |
|---|---|---|
| Hundreds of tools, each task uses a few, and wrong tools get picked | Tool search (deferred tool loading) | Only relevant definitions reach the model |
| Long procedures, each needed only for certain tasks | Skills | Instructions load only when the task calls for them |
| Small, stable content needed on nearly every request | Monolithic, with prompt caching | No lookup step; caching makes it cheap |
| Large reference data read occasionally | MCP resources or file browsing | Fetched on demand instead of preloaded |
Domain 4: Evaluation, Testing & Optimization 16%
4.1 · Define evaluation metrics (accuracy, latency, cost, safety, security)
Rule: If you cannot say what “good” is as a number, define the metric and target before changing anything; if a stem mentions perceived speed, think time to first token, and if it mentions outside content, add a security metric.
| When the stem says… | Pick | Why |
|---|---|---|
| Streamed chat “feels slow” though average time meets target | Time to first token and p95 latency | Users feel the wait for the first words and remember the slow tail |
| A nightly job nobody waits on | Throughput and cost per task | First-token speed does not matter when no one is watching |
| An agent makes many calls per task | Success rate and cost per completed task | Per-call numbers hide retries and loops |
| Open-ended answers with many good wordings | Rubric score from a judge or a person | Exact match would mark good paraphrases wrong |
| The system reads content from outside the company | Add injection success rate and leakage rate | Accuracy metrics say nothing about attacks |
4.2 · Design evaluation datasets and test frameworks using mixed methodologies
Rule: If a check is objective, grade it with code; if it is open-ended and high-volume, use a rubric judge calibrated on human labels; if it is high-stakes or low-volume, have a person review it.
| When the stem says… | Pick | Why |
|---|---|---|
| Output must be valid JSON with fixed fields | Code checks | Exact, instant and free |
| Thousands of open-ended outputs a week | LLM-as-judge with a rubric, calibrated on human labels | Scales and still grades meaning |
| Errors are costly (medical, legal) | Human expert review on top of automated checks | The stakes justify the cost |
| The judge disagrees with experts | Fix the rubric and recalibrate before using it | An uncalibrated judge measures the wrong thing |
| Low volume and rare changes | Human review alone can be enough | Automation is not worth building yet |
4.3 · Conduct A/B testing and iterative improvements
Rule: If a change has not passed the offline eval, run it there first; if the question is business impact, A/B test one change on a random share of traffic with guardrails agreed up front.
| When the stem says… | Pick | Why |
|---|---|---|
| A new prompt looked better on a few examples | Run the full offline eval first | Anecdotes miss regressions |
| Offline eval passed; the question is business impact | A/B test on a random share of live traffic | Only real users show conversions or resolutions |
| Showing users a worse answer is unacceptable | Shadow test | Live inputs, no user exposure |
| Prompt and model both changed at once | Test them separately | Otherwise the cause is unknown |
| Traffic is too low for an A/B result in reasonable time | Offline eval, then staged rollout with close monitoring | An A/B test would never reach a clear answer |
4.4 · Diagnose system issues (prompt failure, hallucinations, model mismatch)
Rule: If answers are wrong, look at the inputs and retrieved context first, then the prompt, then the model; if the fact is missing, fix retrieval, and if it is present but ignored, fix the prompt.
| When the stem says… | Pick | Why |
|---|---|---|
| Confident wrong answers right after a document refresh; model unchanged | Check retrieval and indexing | The data changed, the model did not |
| Similar requests get different formats or handling | Fix the prompt (clear rules, examples) | Instructions are ambiguous |
| The right passage was retrieved but the answer contradicts it | Ground the answer: quotes, citations, allow “I don't know” | Context is fine; the instruction to use it is weak |
| Context right, prompt clear, multi-step reasoning still fails | Try a more capable model | Upstream causes are ruled out |
| A simple task is accurate but expensive | Try a smaller, faster model | Mismatch the other way: overpaying |
4.5 · Optimize token usage, latency, and cost-performance trade-offs
Rule: If nobody waits for the result, batch it; if the same prefix repeats, cache it; if independent steps run in series, parallelise them; and if quality is not re-checked, do not ship the optimisation.
| When the stem says… | Pick | Why |
|---|---|---|
| A large job nobody waits on | Message Batches API (plus a smaller model if the eval holds) | About half price; latency does not matter |
| The same long instructions on every request | Prompt caching | The prefix is paid in full once, then read cheaply |
| Independent steps run one after another | Parallel calls | Total wait becomes the longest step |
| Chat feels slow and answers are long | Streaming and shorter outputs | First words sooner, less to generate |
| A user needs the answer within seconds | Real-time calls with caching or a smaller model, not batch | Batch can take hours |
4.6 · Monitor system performance using logging and observability tools
Rule: If you cannot tell which change caused a drop, log prompt and model versions per request; if errors look normal but users complain, score sampled outputs for quality.
| When the stem says… | Pick | Why |
|---|---|---|
| Quality dropped after a release; cause unknown | Group quality by prompt and model version | Shows which change caused the step |
| An agent is slow and nobody knows which step | Per-step tracing | Finds the slow step directly |
| Cost rising while traffic is flat | Tokens per request and cache hit rate | Points to bigger prompts or a broken cache |
| Wrong answers but error rate normal | Sampled online quality scoring | Bad content still returns success |
| A severe harmful output is live right now | Roll back first, then investigate with the logs | Stopping harm comes before diagnosis |
Domain 5: Governance, Safety & Risk Management 14%
5.1 · Implement guardrails and safety controls
Rule: If a rule must always hold, enforce it in code or permissions; if it is a soft preference, guide it in the prompt; and assume untrusted content will sometimes inject, so limit what the tools can do.
| When the stem says… | Pick | Why |
|---|---|---|
| A rule that must always hold (a refund ceiling, approved recipients) | Enforce it in tool code or permissions | Deterministic; a prompt is followed most of the time, not always |
| Untrusted content reaches the model (web pages, emails, uploads) | Treat it as data and limit what the tools can do | Some injection will get through, so limit the damage it can cause |
| A tone, style or scope preference that does little harm if missed | Guide it in the system prompt | Here the prompt wins: code enforcement would be heavy for a soft rule |
| Sensitive data must never appear in responses | Keep it out of the context, and redact output as a backstop | Data the model never sees cannot leak |
| Cost or volume could spiral | Rate limits, loop caps and budget limits in code | A hard ceiling stops runaway behaviour |
5.2 · Identify risks, limitations, and failure modes of LLM systems
Rule: If the output must be exact or current, get it from a tool or code; if it must be grounded, require sources; if an agent can loop, cap it in code.
| When the stem says… | Pick | Why |
|---|---|---|
| Exact maths over many figures is sometimes wrong | A calculation tool or code | Asking for precision or step-by-step reasoning makes errors rarer, not impossible |
| Answers about recent events or today's prices are out of date | Retrieval or a live-data tool | The model's knowledge stops at its training cutoff |
| Answers contain facts that are not in the documents | Ground in retrieved text, require citations, allow “I don't know” | Targets hallucination at its cause |
| The judgement, not the arithmetic, is what goes wrong | Chain-of-thought or extended thinking | Here the runner-up wins: a calculator cannot fix a wrong judgement |
| An agent repeats tool calls for hours | Iteration and budget caps in code | Stops runaway cost whatever the model decides |
5.3 · Apply human-in-the-loop validation strategies
Rule: If an action is irreversible or high-impact, require approval first; if a case fails a check, send it to review; if it is low-risk, automate it and audit a random sample.
| When the stem says… | Pick | Why |
|---|---|---|
| The action cannot be undone or is high-value | Human approval before it executes | A later audit cannot reverse the harm |
| High volume, low impact, mostly correct | Automate, and audit a random sample | Reviewing everything would not scale |
| The case failed a check or lacks evidence | Send that case to a reviewer | Focuses human time where errors are likely |
| Reviewers approve 99% of items in seconds | Redesign the review: smaller risk-based queue, evidence shown, spot checks | Rubber-stamping means the control has stopped working |
| Every output carries legal or safety weight | Human review of every item | Here the runner-up wins: the risk justifies the cost |
5.4 · Ensure compliance with regulations (e.g., GDPR, HIPAA, FedRAMP)
Rule: If the data is EU personal data, minimise it and control purpose, region, retention and erasure; if it is US health data, use a BAA-covered deployment with minimum necessary PHI and audit logs; if it is US federal work, use a FedRAMP-authorised deployment.
| When the stem says… | Pick | Why |
|---|---|---|
| EU customers' personal data | Minimise, keep to the stated purpose, honour access and erasure, control region and retention | GDPR's core demands |
| Patient records at a US hospital | A deployment covered by a BAA, minimum necessary PHI, access control, audit logs | HIPAA's core demands |
| A US federal agency workload | A deployment with the appropriate FedRAMP authorisation | Federal use requires the authorisation |
| A person asks for their data to be erased | Erase it everywhere, including logs and eval copies | Erasure covers every copy, not just the main database |
| The task works on aggregate or properly de-identified data | Use that data instead of raw records | Here the lighter path wins: data that is not personal or PHI carries far fewer obligations |
5.5 · Address ethical AI considerations (bias, fairness, transparency)
Rule: If fairness is in doubt, measure outcomes across groups and hunt for proxies; if a decision is consequential, a named person owns it; if people meet the AI, tell them and offer a person.
| When the stem says… | Pick | Why |
|---|---|---|
| Leadership worries the system may be biased | Measure outcomes across groups on a representative test set | Evidence first; you cannot fix what you have not measured |
| A protected attribute was removed but outcomes still differ | Look for proxies, then mask and re-measure | Other fields can stand in for the attribute |
| People do not realise a reply was AI-generated | Disclose AI use, say who decides, offer a person | Transparency builds trust and prevents false beliefs |
| The decision affects someone's job, credit or care | A named person makes and owns the decision | Accountability cannot sit with a model |
| A low-stakes internal draft, such as meeting notes | Light disclosure; no group-outcome study | Here the lighter option wins: little to measure and little harm |
Domain 6: Stakeholder Communication & Lifecycle Management 14%
6.1 · Conduct structured discovery and requirement gathering
Rule: If success is not yet defined, run discovery and agree a measurable target first; if success is defined but the data is unchecked, assess the data and prototype; if both are settled, move on to design.
| When the stem says… | Pick | Why |
|---|---|---|
| A sponsor asks for a technology (“a chatbot”, “agents”) with no defined outcome | Discovery: goal, users, measurable success criteria | The request is a solution, not a problem |
| Stakeholders want different things | Bring them together to agree one measurable outcome first | Designing for competing goals satisfies no one |
| Success is defined, but nobody has checked the data | Assess the data, then prototype on a real sample | Missing or stale data is the most common project killer |
| Goal, data and constraints are clear and signed off | Move on to design | Here more discovery would only delay the project |
6.2 · Communicate architectural decisions and trade-offs
Rule: If the listener is an executive, speak in value, cost and risk; if they are engineers, speak in design detail; if someone else owns the decision, give options, trade-offs and your recommendation, then record it in an ADR.
| When the stem says… | Pick | Why |
|---|---|---|
| An executive questions the cost of the design | Cost per correct outcome, with eval numbers and options | Speaks to the decision they own |
| A board or sponsor must decide in a short meeting | Value, risk, cost and a recommendation first; detail on request | They need what the decision depends on |
| Engineers will build and run it | Component and interface detail, failure handling | They need the how, not just the why |
| A decision is likely to be questioned again later | Write an ADR | Keeps the context and reasons after people move on |
| Only one option meets a hard constraint (data must stay in region) | State the constraint and the single option | Here inventing alternatives would waste time |
6.3 · Manage stakeholder feedback loops and expectation alignment (including SLAs)
Rule: If stakeholders expect perfection, agree a measured error target plus error handling; if targets are vague, make them measurable SLAs; if scope grows, assess the impact and agree a revised plan openly.
| When the stem says… | Pick | Why |
|---|---|---|
| Stakeholders expect 100% accuracy | A measured target plus error handling, shown with eval data and the human baseline | Perfection is not achievable; a managed error rate is |
| Sponsor and team disagree whether it is “fast enough” | Agree measurable SLAs and track them | A vague target cannot settle the argument |
| A new group or feature is requested mid-project | Assess impact on cost, time and quality, then agree a revised plan | Silent absorption breaks timelines and quality |
| Users complain informally and nothing changes | A regular review that turns feedback into eval cases | Closes the loop |
| Errors would cause serious harm (medication doses) | Human check of every output, even with less automation | Here the stricter option wins: error handling must cover every case |
6.4 · Document architectures and provide implementation guidance
Rule: If a team must change the system, give it versioned prompts and the eval suite; if it must run the system, give it runbooks and named owners; if it must understand the system, give it the architecture and ADRs.
| When the stem says… | Pick | Why |
|---|---|---|
| The new team fears changing prompts without knowing the effect | The eval suite tied to versioned prompts | Proves a change helps, and allows rollback |
| On-call staff have never handled this system's incidents | Runbooks with clear steps | Nobody should rely on memory under pressure |
| Nobody knows who owns an integration | An ownership table | Incidents stall without a named owner |
| The new team asks why a design choice was made | ADRs | Record the context and the rejected options |
| A two-week prototype that will be thrown away | A short summary of findings | Here the light option wins: a full pack would be wasted effort |
6.5 · Support lifecycle phases (discovery, design, handoff, monitoring, iteration)
Rule: If a new model version arrives, test it on the eval suite and roll it out in stages; if quality drifts with no code change, look at data and usage and add the new cases to the eval suite; if a version is retiring, migrate early.
| When the stem says… | Pick | Why |
|---|---|---|
| A newer model version is available | Eval suite on it, compare quality and cost, then a staged rollout | Newer is not automatically better for your task |
| Quality slowly falls with no code or prompt change | Look at data and usage drift, then iterate | The world moved, not the code |
| The current model version will be retired in some months | Test the new version now and migrate in stages before the date | Avoids a rushed, untested switch |
| A production failure was found | Fix it and add it to the eval suite | Stops the same failure coming back |
| The new version is better and cheaper on every eval metric | Stage the rollout quickly | Here moving fast wins, but still in stages |
Domain 7: Developer Productivity & Operational Enablement 7%
7.1 · Configure Claude tools and environments for teams (e.g., Claude Code)
Rule: If a rule must hold for everyone and nobody may loosen it, use managed settings; if it is the team's setup, commit it to the repo; if it must always run or be blocked, use a hook or permission rule, not CLAUDE.md.
| When the stem says… | Pick | Why |
|---|---|---|
| An organisation-wide rule nobody may loosen | Managed settings | Highest precedence; users cannot override |
| A team convention, such as the test command or style | Project CLAUDE.md | Shared guidance that travels with the repo |
| Something must happen or be blocked every time | Hook or permission rule in .claude/settings.json | Enforced, not just suggested |
| The team shares an MCP server that needs a token | .mcp.json with an environment-variable reference | Shared setup, no secret in git |
| One repository needs a restriction its own team owns | Deny rule in that project's settings | Managed settings are for organisation policy, not one team's choice |
7.2 · Improve developer workflows using AI-assisted tooling
Rule: If the team wants the same check on every change, run Claude in CI with structured output; if someone asks whether it pays off, measure cycle time and defects, and always keep humans approving.
| When the stem says… | Pick | Why |
|---|---|---|
| A consistent automated review on every pull request | Claude Code in CI, non-interactive, structured output | Runs the same way every time without relying on people |
| A developer wants help before they push | Local Claude Code with a shared review command | Fast feedback for one person |
| Leadership asks if it is worth the cost | Cycle time and escaped defects before and after | Outcomes, not usage |
| Hundreds of files need the same library upgrade | Claude-driven migration in small batches, tests on each | Repetitive and checkable |
| Someone suggests auto-merging what Claude approves | Keep human approval | The first pass is not the decision |
7.3 · Support debugging and operational issue resolution
Rule: If Claude is investigating, give it read-only access to logs, metrics and runbooks; if it proposes a change to production, a human approves and applies it; if a bug is intermittent, find the root cause before fixing.
| When the stem says… | Pick | Why |
|---|---|---|
| On-call wants help triaging incidents | Read-only MCP access to logs, metrics and runbooks | Fast investigation with no production risk |
| Claude proposes a production change | Engineer approves and applies it through the change process | High-impact actions need a human and a record |
| A test fails intermittently | Reproduce, find the root cause, then fix | Retries hide the bug |
| A low-risk, well-tested runbook step runs many times a week | Consider approved automation for that step only | Approval for every repeat adds little safety |
| Engineers paste logs into a chat by hand | Connect the sources through MCP | Faster and complete, with less copying |