Exam CCAR-P Topic 1 Question 35 Discussion

An evaluation set must represent the populations, operating conditions, and use cases that the deployed system will encounter. A dataset drawn from only one geographic region and one tenure band cannot reliably demonstrate that the hiring-support tool performs consistently across its intended users. Expanding the evaluation set enables architects to identify regional performance differences, tenure-related errors, unsupported generalizations, and potentially discriminatory outcomes before broader deployment. The system should then be rescored using the expanded dataset, with results examined across relevant population slices rather than relying only on an aggregate score. Options A and B preserve or worsen the representation gap.
Option D is also inappropriate because qualitative feedback can supplement structured evaluation but cannot replace measurable, repeatable testing of accuracy, fairness, and reliability across the intended deployment population.