Towards Chunk-less RAG
summary
Extracts token vectors from Qwen3-Embedding-0.6B and computes cosine between every query and document token to produce relevance heatmaps, clusters high-scoring tokens with Gaussian smoothing and max-peak scoring to yield spans, and evaluates them (strong F1 against ground-truth passages in the 14-page SpongeBob laptop doc). Method outperforms ColBERT variants on these tests but inflates storage by ~900× storage and shows decoder-caused bias toward sentence and document ends, so it’s experimental not production-ready.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:50Limitations of chunk-based RAG
- 1:53Long context embedding models
- 2:35Example document overview
- 3:38Token-level embedding extraction
- 4:41Relevance matrix visualization
- 5:45Noise in relevance signals
- 6:46Causal attention bias explanation
- 7:49Algorithm for relevant spans
- 9:14Results on example queries
- 9:58More query results
- 10:41Drawbacks and challenges
- 12:04ColBERT model comparison
- 13:49Conclusion and next steps
- 14:51Q&A session
