Structured extraction mind maps
Everything the CCA-F exam asks about pulling structured data out of messy documents, on one page. It starts with which exam scenarios bring these questions. Then six mind maps, each in the form "what you see, where the fault is, the fix", with the rule to remember, the exam-guide line it comes from, and one practice question.
Domain 4 · 20%Domain 5 · 15%Guide 4.1–4.5, 5.1, 5.5About 30 minutes
Which scenarios ask about extraction?
The exam shows you 4 scenarios out of a bank of 6. One is about extraction directly. Two others reuse the same skills:
| # | Scenario | Domains | Extraction questions? | Maps to study |
|---|---|---|---|---|
| 6 | Structured Data Extraction | D4, D5 | Yes, the core one. Schemas, tool use, validation, retries, edge cases, human review | All six |
| 5 | Claude Code for Continuous Integration | D3, D4 | Partly. JSON findings, explicit criteria, few-shot, batch for overnight jobs | 1, 4, 5 |
| 1 | Customer Support Resolution Agent | D1, D2, D5 | A little. Keeping exact facts (amounts, dates) in a “case facts” block | Trap 2 below |
| 2 | Code Generation with Claude Code | D3, D5 | No | — |
| 3 | Multi-Agent Research System | D1, D2, D5 | No (sources and citations instead) | — |
| 4 | Developer Productivity with Claude | D2, D3, D1 | No | — |
Guide §5, scenario 6: “Structured Data Extraction — extract from unstructured docs, validate via JSON schemas, handle edge cases. (D4, D5)”
First, the picture: the extraction pipeline
The six maps at a glance
| # | Question | Usual answer | Practise |
|---|---|---|---|
| 1 | How do you get JSON that always parses? | A tool whose input is your JSON schema, with the right tool_choice | tool_choice |
| 2 | Claude makes up values. How do you design the schema? | Optional or nullable fields, and "unclear" or "other" plus detail in enums | Extraction honesty |
| 3 | Validation failed. Retry? | Retry with the exact error, unless the data isn't in the document | Extraction honesty |
| 4 | Output is inconsistent across documents | 2 to 4 few-shot examples. Explicit criteria when the instruction is vague | Few-shot, Explicit criteria |
| 5 | Thousands of documents. Batch or now? | Batch when nobody waits. Resubmit only the failures by custom_id | Batch or now |
| 6 | Can we stop human review? | Only after checking accuracy by document type and field | Confidence and sampling |
A yellow row in each map is the runner-up: the case where a different answer wins because one fact in the question changes.
Mind map 1: How do you get JSON that always parses?
| What you see | Where the fault is | Fix |
|---|---|---|
| Sometimes the JSON is broken (a missing bracket, extra text around it) | Claude is asked for JSON in plain text | Define a tool whose input schema is your JSON schema and read the tool call |
| Claude sometimes replies in text instead of calling the tool | tool_choice is "auto", which allows text | tool_choice: "any" (must call a tool) |
| There is one extraction tool and it must always run | auto may skip it | Force it: {"type":"tool","name":"extract_invoice"} |
| There are several schemas (invoice, receipt, contract) and the document type is unknown | Forcing one tool picks the wrong schema for other types | "any", so Claude picks the right extraction tool |
| The JSON is valid, but line items don't add up to the total | A schema checks shape, not meaning | Add your own validation after extraction (map 3) |
any when any of several tools is fine, and force a named tool when it must be that one. A strict schema removes syntax errors, not wrong values.Guide 4.3: “tool_use with JSON schemas = most reliable schema-compliant output (eliminates JSON syntax errors). tool_choice: "auto" (may return text), "any" (must call a tool), forced (specific named tool). Strict schemas eliminate syntax errors but not semantic errors.”
tool_choice: "auto"with a stronger system prompttool_choice: "any"- Forced
tool_choiceforextract_invoice - Ask for JSON in the prompt and parse the text
Mind map 2: Claude makes up values. How do you design the schema?
| What you see | Where the fault is | Fix |
|---|---|---|
A field like due_date is filled in even when the document has none | The field is required, so Claude must put something there | Make it optional or nullable |
| A category is forced into the nearest wrong value | The enum has no way out | Add "unclear" or "other" to the enum |
Many documents land in "other" and you can't tell why | “Other” has no detail | Add "other" plus a detail field, like category_detail |
| The field is always present in this document type (like an invoice number) | Making it optional hides real extraction misses | Keep it required |
unclear). A required field with no source value invites a made-up answer.Guide 4.3: “Optional/nullable fields prevent fabrication; enum "unclear"/"other" + detail for extensible categories.”
- Tell Claude in the prompt not to guess dates
- Make
renewal_datenullable in the schema - Retry whenever the renewal date equals the signing date
- Add a few-shot example of a contract without a renewal date
Mind map 3: Validation failed. Retry?
First ask one question: is the information in the document? If it is, a retry can fix it. If it isn't, no retry will.
| What you see | Where the fault is | Fix |
|---|---|---|
| A date is in the wrong format, or a field is in the wrong place | A format or structure mistake | Retry with the specific validation error added to the request |
| Retries keep failing the same way | The retry just says “try again” | Send the exact error (“line items sum to 410, total says 450”) |
| The value is simply not in the document | Retrying can't create it | Don't retry. Return null or route to a human |
| Totals don't match and you can't tell if Claude or the document is wrong | Only one number is extracted | Extract calculated_total and stated_total, with a conflict_detected flag |
| Reviewers keep dismissing the same kind of flag | You can't see which pattern caused it | Record a detected_pattern field and analyse the dismissals |
Guide 4.4: “Retry-with-error-feedback: append specific validation errors on retry. Retries are ineffective when info is absent from the source. Self-correction: extract calculated_total alongside stated_total; conflict_detected booleans.”
invoice_date came back as “3rd March 2026” instead of YYYY-MM-DD. What should happen?
- Send it to a human reviewer
- Retry, including the error “invoice_date must be YYYY-MM-DD, got ‘3rd March 2026’”
- Retry with the same request
- Make
invoice_datenullable
Mind map 4: Output is inconsistent across documents
| What you see | Where the fault is | Fix |
|---|---|---|
| Clear instructions, but output still varies from document to document | Instructions alone aren't enough | 2 to 4 few-shot examples of input and wanted output |
| Ambiguous cases are handled differently each time | No example shows the edge case | Examples that show the ambiguous case and the reasoning for the choice |
| Documents come in very different layouts | Examples cover only one layout | Examples from varied document structures |
| Claude invents values that look plausible | Nothing shows what “not there” looks like | A few-shot example with a missing value returned as null |
| The instruction itself is vague (“flag anything suspicious”) | Claude doesn't know what counts | Explicit categorical criteria first. Examples then show them in use |
Guide 4.2: “Few-shot is the most effective technique for consistent, actionable output when instructions alone are inconsistent. Demonstrate ambiguous-case handling; reduce extraction hallucination. 2-4 targeted examples showing reasoning for the chosen action; handle varied doc structures.” Guide 4.1: “Explicit criteria beat vague instructions.”
- Add “be careful to pick the tax, not the subtotal” to the prompt
- Add three examples from different shop layouts, each showing which line is the tax and why
- Use a larger model
- Retry whenever tax is more than 30% of the total
Mind map 5: Thousands of documents. Batch or now?
| What you see | Where the fault is | Fix |
|---|---|---|
| A nightly or weekly load, and nobody waits for the result | Paying full price for speed nobody needs | Message Batches API: 50% cheaper, results within 24 hours |
| You need to match each result to its document | Results don't come back in order | Give each request a custom_id |
| Some documents in the batch failed | Resubmitting everything costs twice | Resubmit only the failures, by custom_id |
| A user uploads a document and waits on screen | Batch has no speed guarantee | Real-time requests |
| Extraction needs several rounds of tool calls (look up, then extract) | One batch request can't do a multi-turn tool loop | Real-time, or split into separate steps |
custom_id, and resubmit only what failed.Guide 4.5: “Message Batches API: 50% cost savings, up to 24-hour window, no latency SLA. Batch API does not support multi-turn tool calling in a single request. custom_id correlates request/response; resubmit only failed docs by custom_id.”
- Rerun the whole batch
- Resubmit the 300 by
custom_idin a new batch - Process the 300 in real time
- Send the 300 to human review
Mind map 6: Can we stop human review?
| What you see | Where the fault is | Fix |
|---|---|---|
| Overall accuracy is 97%, so the team wants to stop reviewing | An average hides weak document types and fields | Measure accuracy by document type and by field first |
| You don't know how often high-confidence results are wrong | Nobody checks the confident ones | Stratified random sampling, including high-confidence results |
| You need to decide which results go to a human | There's no trusted signal to route on | Field-level confidence, calibrated on a labeled set |
| Routing uses Claude's own “I'm 90% sure” with no calibration | Self-reported confidence isn't reliable | Calibrate it on labeled data before relying on it |
Guide 5.5: “Aggregate accuracy (97%) can mask poor performance on specific doc types/fields. Stratified random sampling measures high-confidence error rates; field-level confidence calibrated on labeled sets routes review. Validate accuracy by document type and field before automating.”
- Remove review, since 98% beats the target
- Measure accuracy separately for each document type and field, including handwritten forms
- Review only results Claude marks as low confidence
- Keep reviewing everything
Three traps that catch people
1 · Syntax or meaning?
A strict schema guarantees the JSON parses and has the right fields. It doesn't guarantee the numbers are right. If the question says “valid JSON but wrong totals”, the answer is validation, not a stricter schema.
2 · Long documents and exact facts
Claude reads the start and end of a long input best and may miss the middle. Put key facts at the start with clear headings, and keep exact values (amounts, dates, IDs) in a separate facts block instead of a summary that turns “$1,240 on 3 March” into “a refund last month”.
3 · JSON: the exam's answer vs today
Today's API also has other structured-output options, but the guide teaches “tool with a JSON schema” as the most reliable. On the exam, follow the guide. More on Exam vs 2026, item 1.
Guide 5.1: “‘Lost in the middle’: models reliably process beginning/end but may omit middle sections. Extract transactional facts into a persistent ‘case facts’ block.”
How to study this
Related: Domain 4 hub · Domain 5 hub · Claude Code mind maps · MCP mind maps · Exam vs 2026 · all traps · CCA-F start here