10 (funny) years of Apache Lucene hacking
summary
Ten years of Lucene/Solr hacking traces concrete changes: replacing millions of short Token objects with tokenstream attributes to cut allocation and speed indexing, adding doc values for fast sorting, shipping flexible codecs so custom compression and file formats plug in, and deploying finite state transducers to run regex/wildcard queries much faster. Also documented are painful Java 7 VM/compiler bugs, memory‑mapped index GC crashes, and a shift to randomized testing and static “policeman” checks that catch many subtle bugs.
AI-written summary · about
chapters (9)
- 0:00Introduction to decade of Lucene
- 3:07Early days and community growth
- 8:30Codec evolution and compatibility
- 15:24Token streams and performance
- 21:13Java 7 hotspot era challenges
- 25:40Memory mapping and JVM issues
- 30:06Randomized testing and solutions
- 35:36Latest features and future plans
- 36:55Q&A and closing remarks