Exam CCAR-F Topic 1 Question 50 Discussion
Actual exam question for Anthropic's CCAR-F exam
Question #: 50
Topic #: 1
Question #: 50
Topic #: 1
You are building a structured data extraction system using Claude. The system extracts information from unstructured documents, validates the output using JavaScript Object Notation (JSON) schemas, and maintains high accuracy. It must handle edge cases gracefully and integrate with downstream systems.
Monitoring shows 12% of extractions fail Pydantic validation with specific errors like "expected float for quantity, got '2 to 3'". Retrying these requests without modification produces identical failures.
What's the most effective approach to recover from these validation failures?
Monitoring shows 12% of extractions fail Pydantic validation with specific errors like "expected float for quantity, got '2 to 3'". Retrying these requests without modification produces identical failures.
What's the most effective approach to recover from these validation failures?
Suggested Answer: A Vote an answer
An unchanged retry repeats the same task specification and therefore commonly reproduces the same invalid interpretation. The validator has generated precise corrective information- quantity requires a float, but the model returned the range string 2 to 3 . Supplying that error in a follow-up turn converts a generic retry into an iterative repair operation.
Anthropic identifies iterative refinement as a method for detecting and correcting inconsistencies by feeding an earlier output back into a subsequent request with targeted instructions. ( https://docs.anthropic.com/en/docs
/test-and-evaluate/strengthen-guardrails/reduce-hallucinations ) Option A applies that pattern directly. Claude receives the invalid output, the exact Pydantic error, and an instruction to return a schema-compliant correction. The application should cap retries, retain the original source, and escalate cases that cannot be represented without information loss.
Option B does not guarantee correct formatting; lower temperature may make the same wrong output more repeatable. Option C can be valuable for systematic OCR or source-format problems, but it is unnecessarily broad when the immediate failure is already described by the validator. Option D increases cost and complexity without first using the actionable feedback available from the existing validation layer.
For supported models, native Structured Outputs should also be considered because they guarantee schema- conformant JSON and can prevent many Pydantic shape failures before they occur. ( https://platform.claude.
com/docs/en/build-with-claude/structured-outputs )
Official references/topics: Iterative Refinement; Validation-Error Feedback; Bounded Retry Loops; Structured Outputs.
Anthropic identifies iterative refinement as a method for detecting and correcting inconsistencies by feeding an earlier output back into a subsequent request with targeted instructions. ( https://docs.anthropic.com/en/docs
/test-and-evaluate/strengthen-guardrails/reduce-hallucinations ) Option A applies that pattern directly. Claude receives the invalid output, the exact Pydantic error, and an instruction to return a schema-compliant correction. The application should cap retries, retain the original source, and escalate cases that cannot be represented without information loss.
Option B does not guarantee correct formatting; lower temperature may make the same wrong output more repeatable. Option C can be valuable for systematic OCR or source-format problems, but it is unnecessarily broad when the immediate failure is already described by the validator. Option D increases cost and complexity without first using the actionable feedback available from the existing validation layer.
For supported models, native Structured Outputs should also be considered because they guarantee schema- conformant JSON and can prevent many Pydantic shape failures before they occur. ( https://platform.claude.
com/docs/en/build-with-claude/structured-outputs )
Official references/topics: Iterative Refinement; Validation-Error Feedback; Bounded Retry Loops; Structured Outputs.
by Monroe at Jul 23, 2026, 05:37 PM
0
0
0
10
Comments
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
Report Comment
Commenting
You can sign-up / login (it's free).