Why a 1990s Machine Learning Algorithm Destroys LLMs at...
summary
Trains a Random Forest in Ruby with Rumale on ~23,000 Las Vegas parcel records (70/30 split), serializes the model and scores it on 120 test cases using MAE and RMSE; the Random Forest beats Claude 4.6 on accuracy, runs in <1 ms versus >2 s for the LLM, and yields near-zero variance across 120 repeats while the LLM shows high inconsistency and higher cost. Builds a hybrid pipeline where the LLM extracts parameters and the Random Forest predicts — it works and reduces latency, variance, and billable inference.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:47LLM house hunting frustration
- 3:29Why old algorithms matter
- 4:13Random forest algorithm
- 7:25Ruby ML gem for random forests
- 9:10Training random forest model
- 10:34Experiment setup vs LLMs
- 11:56Accuracy results in test
- 12:59Latency and consistency
- 13:41Why LLMs fail consistency
- 15:03Hybrid pipeline with LLMs
- 17:28LLMs vs specialized tools
- 19:15Hybrid dream home search
- 20:18Wrong with rigor vs right by accident
- 20:39Conclusion
related talks
35mIntelligent Ruby: Getting Started with Machine Learning
25mIntelligent Data Classification with Ray and vLLM at Apple
50mKa-Ching! How Enterprises Measure LLM Performance and Cost
51mMachine Learning with Ruby
52mReduce LLM calls with Vector Search Design Patterns
21mMachine Learning with Julia
37mAutomated Machine Learning: Building a Pipeline
20mEverything I Learned Training Frontier Small Models