Use these questions to practice explaining retrieval evaluation like someone who has operated a RAG system, not only read about one.
Because a bad answer may be caused by missing evidence, buried evidence, noisy evidence, stale evidence, or generation behavior. Retrieval metrics isolate the evidence layer before the model writes anything.
It is the fraction of expected gold evidence retrieved within the top K results. It answers: did the retriever find the evidence early enough to be available?
It is the fraction of top K retrieved results that are relevant. It helps detect noisy final context that can distract the model or waste tokens.
MRR tells you how early the first relevant result appears. Recall can look good even when the correct evidence is ranked too low for the final context.
Use it when relevance is graded. A direct policy clause should score higher than a background FAQ, and nDCG rewards ranking stronger evidence earlier.
Gold evidence is the labeled source, chunk, span, or table row expected to support an answer. It turns retrieval evaluation from opinion into measurement.
Inspect corpus coverage, parsing, chunking, embeddings, sparse search, query rewriting, metadata filters, and index freshness. Low recall means candidate discovery is weak.
The system finds the evidence but ranks it too low. Inspect fusion, reranking, dedupe, final-k cutoff, and whether query rewriting is changing intent.
Averages hide pain. Exact-term support, PDF tables, legal policies, and finance workflows can fail differently even when overall Recall@10 looks healthy.
It is relevant evidence removed by metadata, access, tenant, freshness, or policy filters. It should be reported separately because the fix may be filter logic, not search quality.
Recall@K, Precision@K, MRR, nDCG where needed, hit rate, latency, empty retrieval rate, filter drop rate, stale hits, source freshness, index version, and route version.
Define thresholds by risk and slice. A high-risk legal or finance regression should block even if the average metric improves.
It can hide that the evidence is ranked too low for the final prompt. If your answer uses top 5, Recall@50 may give false comfort.
Retrieval metrics explain whether the answer had enough source support. Answer evaluation then checks faithfulness, completeness, tone, refusal behavior, and citation quality.
I would label gold evidence, evaluate retrieval before generation, compute multiple metrics, slice them by workflow and risk, inspect failures through traces, and use the results as release gates tied to concrete engineering actions.