Practice Bank: Why RAG Exists
This practice bank is built for production judgment. Each scenario asks you to decide whether RAG is appropriate, what evidence should be retrieved, and what failure should become an eval case.
Product Scenarios
HR Policy Assistant
An employee asks about parental leave. Decide the source policy, citation requirement, and failure risk.
Customer Support Refund Bot
A refund answer depends on country, product tier, and policy version. Define the metadata needed.
Sales Enablement Assistant
A salesperson asks if they can promise a restricted discount. Identify the approval source and escalation rule.
Legal Contract Assistant
A PM asks whether model-output logging is allowed. Identify what the assistant can answer and what legal must review.
Healthcare Data Assistant
A clinician asks about patient data in a demo. Identify access, compliance, and human-review constraints.
Platform Scenarios
Security Runbook Assistant
An engineer asks what to do after a production API key leak. Decide source, priority, and audit log needs.
Engineering Design Assistant
A platform engineer asks about latency targets. Identify whether this needs docs, metrics, or both.
Finance Compliance Assistant
An analyst asks about account identifiers in a report. Define citation and refusal behavior.
Developer Documentation Assistant
A deploy failed after an environment variable changed. Decide when RAG is enough and when tool use is needed.
Eval Cases
Wrong-Policy Retrieval
The assistant retrieves three chunks from the wrong policy but gives a confident answer. Write the eval case and the first engineering experiment you would run.
1 Your production judgment task
Pick any three scenarios above. For each one, write: source to retrieve, answer rule, refusal rule, eval case, and first metric to inspect.