Exercises: Build a Tiny TF-IDF Searcher
Use starter.py to tokenize documents, build IDF weights, compute TF-IDF vectors, and rank documents.
1 Tokenize text
Lowercase and extract useful word/code tokens without destroying identifiers.
2 Compute IDF
Reward rare terms and reduce the impact of common words.
3 Build TF-IDF vectors
Represent documents and queries in the same vocabulary space.
4 Rank and explain
Return top documents with matched terms and a short trace note.