Project: Chunking Failure Review Board
Build a small review engine that turns bad-answer traces into release-ready failure reports.
What you will build
- A trace classifier that labels chunking failures.
- A severity model that flags critical stale-content and access-risk issues.
- A fix recommender that proposes one controlled change.
- A release report that includes metric, owner, rollback risk, and decision.
Product-level outcome
From complaint to engineering review
Instead of saying "RAG answered badly," your project should output a useful engineering sentence: "Trace legal-003 is a critical stale_chunk failure. Filter active versions, rebuild the index, and prove improvement with stale_hit_rate before release."