100% Big Data. 0% Hadoop. 0% Java.

on YouTube ↗52m

" summary

Pipeline description that decides: streaming Twitter feed ingests into queues, forks into immediate alerts and a store, stores records in react and runs MapReduce on the same nodes via a modified disco; Python NLP via nltk/tweepy trains a Naive Bayes classifier with trigram features and name lists. Numbers: 1.5 million tweets demo, ~7k messages per minute streaming, mapreduce on preloaded store in ~15s. Practical points: queue protects Riak/react, Erlang handles distribution, Rabbit/Zero glue, Hadoop/Java avoided.

AI-written summary · about

" chapters (9)

1. 0:00Start
2. 1:32Big Data for It Decision-Makers
3. 7:37Use Case
4. 16:17Lessons Learned
5. 24:27Sentiment Analysis
6. 27:29Frequently Asked Questions
7. 29:18Why Did I Want To Use It To Run It on the Jvm
8. 30:17Alternatives to the Java Ecosystem
9. 38:03What Did We Learn about Big Data

" related talks

• 01h 26mData Driven Action: a primer on Data ScienceSarah Aerni, Srivatsan Ramanujam, Jarrod Vawdrey · SpringOne2GX 2015
• 01h 25mApache Spark for Big Data ProcessingLudwine Probst, Ilayaperumal Gopinathan · SpringOne2GX 2015
• 30mScalability Challenges in Big Data ScienceMikio Braun · Berlin Buzzwords 2012
• 37mBig Data Analytics, Machine Learning, Real Time ProcessingKai Waehner · Codemotion Milan 2016
• 19mBig DataDaniel Lanza Garcia · Voxxed Days CERN 2017
• 48mHyper Speed! When Big Data BloomsScott Helme · NDC London 2023
• 38mRealtime Data Activation - Analytics, Intelligence & Decision MakingMarcelo Wiermann · THE CONF 2017
• 45mSolving Real & Big (Data) Problems Using HadoopEva Andreasson · GOTO Aarhus 2013