Language as an Interface
summary
Built a one-file JavaScript toolkit called NLP compromise that fits in an npm package about the size of jQuery and a lexicon of roughly 110 kilobytes. Uses WordNet, Zipfian frequency cutoff, suffix regexes and simple sentence-level rules to cut a startup search problem from 10 n-gram queries to 3. Achieves mid-80% POS accuracy, tag-based matching instead of char regexes, averages 8 milliseconds per sentence (a novel can take ~20 seconds), and supports plugin transformations and basic contraction handling.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:45NLP Compromise
- 1:23Being a nonexpert
- 2:15Elastic Search
- 4:02Stop Prepositions
- 5:21Existing Tools
- 6:40How big is language
- 8:38Vocabulary size
- 9:23Word size
- 10:00Wordnet
- 11:35Tomato
- 12:54File Sizes
- 13:27lexicon
- 14:31printout
- 14:51sentence level processing
- 16:03summary
- 19:59most important slide
- 21:17speech tags
- 24:00future projects
related talks
26mThe Meaning of Words
26mDesign a new language in JS: from zero to autocomplete
33mAdvanced Natural Language Processing with Free Low-Code and...
44mDeveloping a Language Plugin: LSP versus the Joy of Learning
27mExecution vs. Parse-Based Language Servers
30mCLI Tools for All the Things
39mWhat Can We Learn With JavaScript Fatigue?
26mParsing Parsers