Fast Analytics On Big Data
summary
Claims an open-source H2O platform that gets 1000x faster than disk and 100x faster than R by keeping data in in-memory compressed columnar vectors split into chunk units, accessed by a single-jar Java map/reduce API that runs from laptop to 100+ servers. It runs GLM/logistic regression, GBM/Random Forest, deep nets and PCA at memory-bandwidth speed, demos a Random Forest on 500k rows in seconds and exports models as POJO model export for production. Limits: no file IO or new threads inside map tasks and no built-in HA.
AI-written summary · about
chapters (20)
- 0:00Intro
- 2:32Big Data Work
- 4:34MapReduce
- 7:12Scholar Version
- 7:39Map
- 8:47Count
- 9:16Set
- 9:53Group Buy
- 10:36Uniques
- 11:21Limitations
- 12:45Strengths
- 14:28How it works
- 16:50Data layout
- 17:43Conceptual view
- 19:54Chunk
- 21:28Alignment
- 22:22Summary
- 23:04Demo
- 37:12Spark Integration
- 43:22Questions
related talks
45mMassive distributed processing with H2O
39mBig Data Systems Performance: The Little Shop of Horrors
41mParallel and distributed big joins in H2O
45mFrom Big Data to Big Information
36mData Warehouse, Data Lakes, What's Next?
51mJava in production for Data Mining Research projects
38mKeynote | Data Warehouse, Data Lakes, What's Next?
07mIn-Hadoop Analytics: Bringing analytics to big data