Unsupported claim
Metric: faithfulness. Action: inspect prompt grounding, context packaging, and unsupported-term patterns.
Practice Bank: Quality Failure Patterns
Learn to name answer failures precisely. The faster you can classify a failure, the faster your team can fix the right layer.
Right topic, wrong citation
Metric: citation correctness. Action: check claim-to-source support, not only citation presence.
Faithful but incomplete
Metric: completeness. Action: add expected conditions to rubric and benchmark labels.
False refusal
Metric: refusal correctness. Action: tune uncertainty threshold and answer-when-supported policy.
Missed refusal
Metric: safety and refusal correctness. Action: strengthen insufficient-evidence behavior and escalation rules.
High score, high severity
Metric: release gate severity. Action: block serious high-risk failures even when averages look good.
Drill Format
For each failure case in the code, return labels, severity, first action, and release decision. This is how answer evaluation becomes an engineering workflow instead of a scoring spreadsheet.