CCAR-F · module 3 of 5 · 20% of the exam

Prompt engineering and structured output

Weight: 20 percent of the exam. Tests whether you can turn a vague prompt into a specified one, and whether you know which guarantees come from the schema and which still have to be earned in code.

What the exam expects

Domain 3 questions sit mostly in three of the six published scenario contexts: prompts for a customer support resolution agent, output formatting for Claude Code running in CI, and structured data extraction from unstructured documents with JSON schema validation.

The recurring architectural judgment is: where does the guarantee come from? An instruction gives you a tendency. A schema gives you a shape. Neither gives you correctness. A large fraction of wrong answers are instructions offered where a mechanism is required, or a mechanism credited with a guarantee it does not provide.

The second judgment is about staging. Detection and filtering, reasoning and formatting, extraction and validation: the exam rewards splitting these into distinct steps and penalizes designs that collapse them into a single self-filtering pass.

Writing the prompt: specificity, structure, and examples

Treat Claude as a capable new colleague with no context on your norms. The documented golden rule: if a colleague with minimal context would be confused by your prompt, so will Claude.

Be explicit about criteria. When a support agent escalates inconsistently on near-identical tickets, the cause is almost never model capability and almost never temperature. It is that "escalate when appropriate" defines no boundary. The fix is criteria a reviewer could check: escalate when the refund exceeds a stated amount, when the account is flagged enterprise, when the customer has contacted support twice about the same order. Raising the model tier just buys a more expensive guess at an unstated policy.

Say what to do, not what to avoid. "Do not use markdown" underperforms "write the reply as smoothly flowing prose paragraphs." Two supporting moves: give the reason behind the instruction, since a model told that its output is read aloud by a speech engine generalizes to punctuation it was never told about; and match your prompt's own formatting style to the output you want.

Use XML tags to draw boundaries. Wrap instructions, context, examples, and variable input in their own descriptive tags. For multiple sources, nest each in a document block with source and document_content subtags inside an enclosing documents block with an index. This is what makes correct attribution possible and keeps a sentence inside a document from being read as an instruction.

Use three to five examples, and make them diverse. Examples are the most reliable lever on tone, format, and structure. The documented guidance is three to five, wrapped in example tags inside an examples block, relevant to the real use case and varied enough that the model does not latch onto an incidental pattern. Four examples that all happen to be billing disputes ending in a refund will teach an agent to steer shipping conversations toward refunds. That is not a bug in few-shot prompting; it is what an undiverse set actually says.

For long inputs (roughly 20,000 tokens and up), placement matters. Put the longform data at the top and the instructions, examples, and query after it, which can improve response quality substantially on complex multi-document inputs. For grounding, ask the model to first pull the supporting quotes into a quotes section and produce its answer from those. Quote-first grounding focuses attention on relevant passages and leaves an auditable trail.

One migration note the exam will test: prefilling the assistant turn is no longer supported on current Claude models and returns a 400 error. Suppressing preambles becomes a direct instruction or a constrained output. Forcing a JSON or classification format becomes structured outputs or a tool with an enum. Continuations move into the user turn.

Structured outputs: what the schema actually guarantees

There are two complementary features.

JSON outputs are requested with output_config.format set to {"type": "json_schema", "schema": {...}}. The response comes back in a text content block as valid JSON matching the schema, with stop_reason of end_turn. This is the stable parameter; an earlier beta used an output_format parameter behind a beta header, and no header is required now.

Strict tool use is requested with "strict": true on an individual tool definition. It applies the same constrained decoding to the tool's arguments, so a call cannot arrive with a missing required field or a string where a number belongs. The distinction the exam presses on: tool_choice controls whether and which tool is called and validates nothing about the arguments; strict mode is what validates them. The two combine freely in one request, with strict tools governing intermediate calls and the output format governing the final response.

Schemas use a subset of JSON Schema. Two requirements apply to every object: additionalProperties must be false, and a required array must list the object's properties. Because every property must be listed in required, an optional field is modeled by permitting null in its type, not by omitting it.

Supported: the basic types, enum over scalars, const, anyOf and allOf with limits, internal $ref and definitions, default, the standard string formats, minItems of 0 or 1, and regex pattern.

Not supported: recursive schemas, external $ref such as a URL, numeric constraints (minimum, maximum, multipleOf), string length constraints, array constraints beyond minItems 0 or 1, complex types in enums, and additionalProperties set to anything but false. An unsupported keyword returns a 400 with details, before any generation happens. Recursion has one correct workaround: flatten to a list of records with parent id references and rebuild the tree in code.

Three operational behaviors round it out. Structured outputs work with streaming, but only the accumulated response is guaranteed valid, so a client must buffer rather than parse each chunk. Changing output_config.format invalidates the prompt cache for that thread, so interleaving six schemas across one cached system prompt destroys your hit rate. And the compiled grammar is cached for 24 hours, which is why the first request after a schema change is slower and the rest of the day is not.

The guarantee boundary is the most examinable fact in this domain. Conformance means valid JSON, required fields present, declared types respected. It says nothing about whether the values are right: cross-field consistency, business rule plausibility, and evaluation remain application concerns. A refusal is also outside the guarantee, coming back as ordinary text, so a production parser must handle a response that is not the expected object and route it to human review.

Automated review: minimizing false positives without losing coverage

This cluster shows up as CI scenarios and has a consistent shape.

The trap is the single self-filtering pass. Telling a reviewer to "report only high severity issues" reliably reduces noise and silently drops real defects, because the model applies the conservative instruction faithfully and nobody ever sees what it suppressed. The documented pattern is the opposite: ask for coverage, including lower-confidence items, then rank and filter in a separate step. Separating recall from precision is the whole design.

Three levers raise precision without touching recall:

1. Explicit inclusion and exclusion criteria. State what counts as a finding (a concrete correctness, security, or data loss defect) and name the out-of-scope categories, with a short example of each. Most "too noisy" complaints are really "nobody wrote down the standard."
2. Evidence requirements. Require each finding to carry a file path, a line number, and the concrete input and execution path that produces the failure, and discard findings that cannot supply them.
3. A second evaluation pass. A separate call scores each candidate against a written rubric and returns a structured verdict per finding, keep or drop, with the criterion that decided it.

That last point connects back to structured output. A pipeline that gates a build should never grep prose for the word "security." Constrain a category and a severity field to enums, or a boolean for whether to open a pull request, and branch on the parsed values. An enum gives the pipeline a value from a closed set, which is exactly what a deterministic gate needs.

Validation loops deserve the same discipline. A good loop retries with the validator's specific error appended so the retry is informed, caps the retries so cost and latency stay bounded, and defines a terminal behavior: publish a clearly labelled unvalidated result, or route to human review. What it must never do is silently repair the payload by inserting defaults, which fabricates data in a system whose only value is trustworthy output.

Batch processing and prompt design at volume

The Message Batches API fits work that is high volume, cost sensitive, and not latency sensitive. All usage is charged at 50 percent of standard prices. Most batches finish within an hour, but the only guarantee is a 24 hour window. Design facts the exam uses:

• A batch is capped at 100,000 requests or 256 MB, whichever comes first.
• Every request carries a custom_id, and results can come back in any order, so results must be joined on custom_id and never zipped by position.
• Result types are succeeded, errored, canceled, and expired. Everything except succeeded is unbilled. expired means the request never reached the model and should be resubmitted; errored means the request was invalid and needs a fix.
• Results are downloadable for a limited period (29 days), so a job that needs them longer must persist them.
• Almost anything you can send to the Messages API can be batched, including system prompts, tool use, extended thinking, and structured outputs. The short unsupported list includes stream: true, which is the usual cause when a copied synchronous request body is rejected wholesale.
• Because batches run longer than the default 5 minute cache lifetime, use the 1 hour cache duration when many requests share a long prefix.

Batching also fixes a prompt design problem. A job that concatenates 4,000 files into one enormous prompt gets inconsistent results and misses content near the end. One request per file (or per small group) with a custom_id per file gives each file the model's full attention, makes results attributable, costs half as much, and lets the shared system prefix stay cached across the run.

How to think through the question

Step 1: name the failure class. Parse errors and malformed shapes are a structured output question. Inconsistent decisions on similar inputs are a criteria question. Values that are well-formed but wrong are a validation or grounding question. Too much noise or a missed defect is a staging question. High volume with a cost complaint is a batch question.

Step 2: ask what the option actually guarantees. For each choice, say out loud what it promises. "Instruct the model to..." promises a tendency. "Constrain with a schema" promises a shape. "Validate downstream" promises semantics. Match the promise to the requirement in the scenario. If the scenario says "must," an instruction is almost never the answer.

Step 3: eliminate anything that hides the failure. Regex repair of malformed JSON, fuzzy-matching an invented label onto the closest valid one, inserting defaults for fields that failed validation, coercing a string amount into a number inside a refund tool. These options are always present and always wrong, because they convert a visible error into a silent one.

Step 4: eliminate the single-pass collapse. Any option that asks one call to both find and filter, or to both reason freely and emit constrained JSON with nowhere for the reasoning to go, is the trap. Look for the option that gives each job its own place.

Step 5: check for the stale technique. Prefill returns a 400 on current models. Structured outputs no longer need a beta header. If an option depends on either, it is wrong regardless of how reasonable the rest of it sounds.

Worked example. To cut noise, a team changed their review prompt to "report only high severity issues." Noise dropped, but a post-incident review found the reviewer had read the code path that caused an outage and said nothing.

Step 1: this is a staging failure, not a capability or context failure. Step 2: the requirement is coverage, and "instruct the model to be selective" promises only a tendency toward selectivity, which is precisely what produced the miss. Step 3: no option here hides a failure, so move on. Step 4: the single-pass collapse is the whole story. The prompt asks one call to detect and to suppress, and suppression happens before any human or any second stage can see the candidate. Step 5: nothing stale. Two distractors survive on plausibility: defining the severity vocabulary is genuinely useful but still filters inside the same pass, and adding repository context does not explain a miss on a path the reviewer demonstrably read. The answer is to ask for full coverage including lower-confidence findings, then rank and filter in a separate step.

Exam traps

• Crediting an instruction with a guarantee. "Return only valid JSON" and "only pick from these nine categories" are tendencies. output_config.format and an enum are guarantees.
• Crediting a schema with correctness. Conformance means shape. Cross-field consistency, business rules, and evaluation are still yours.
• Forgetting the refusal path. A refusal is plain text with stop_reason of end_turn, not schema-conforming JSON, so the parser must expect it.
• Reaching for prefill. Unsupported on current models; returns a 400. Structured outputs, a tool schema, or a direct instruction replace it.
• tool_choice as validation. It forces the call; strict: true validates the arguments.
• Omitting an optional field from required. Every property must be listed; optionality is expressed by permitting null.
• Numeric and length constraints in the schema. minimum, maximum, minLength, and friends are outside the supported subset and return a 400. Enforce them in code.
• Recursive schemas. Not supported. Flatten with parent id references.
• Parsing streamed chunks as JSON. Only the accumulated response is guaranteed valid.
• Varying the output schema on a cached prefix. Changing output_config.format invalidates the cache for that thread.
• "Only report high severity." The single self-filtering pass that under-reports. Coverage first, filter second.
• Truncating to the top N findings. Arbitrary, and it can push a real defect off the list when the ranking criterion is undefined.
• Zipping batch results by position. Order is not guaranteed; join on custom_id.
• Assuming a batch finishes in an hour. Typical, not guaranteed. The window is 24 hours, after which unprocessed requests come back as expired and unbilled.
• One giant prompt instead of many small requests. For per-item work at volume, batch per item.

Quick reference