Exercises: Why RAG Exists
Use starter.py to classify questions as model memory, web search, or RAG. Then push the answer toward production: sources, citations, metadata, refusal behavior, and evals.
starter.py, fill the TODOs, and compare with solution.py.1 Classify the question
Decide whether each question should use model memory, web search, or RAG.
2 Explain your reasoning
For each RAG question, write why trusted context is needed.
3 Add two policy questions
Add one low-risk and one high-risk Northkeep question so you can feel how risk changes the answer design.
4 Add source requirements
For each RAG question, name the document, runbook, contract, or policy that should be retrieved.
5 Add citation behavior
Decide whether the assistant should cite section, page, paragraph, version, approval owner, or all of them.
6 Add refusal behavior
Pick two weak-context situations and write a helpful refusal instead of a confident guess.
7 Add metadata filters
Choose country, department, role, policy version, product tier, or classification filters for three RAG questions.
8 Add a production failure
Create a wrong-document retrieval case and write what you would inspect first.
9 Create a golden eval
Write expected source, expected behavior, bad answer to catch, and the first metric you would inspect.
10 Explain it to a product manager
Write five simple sentences explaining why this product needs RAG instead of only a better prompt.