Building Data Driven Products with Ruby
summary
Builds a production pipeline that turns 10 billion log lines into 10,000 log lines to diagnose a missing-user-id bug, uses Ruby for collection/cleaning (Nokogiri, RestClient, PDF::Reader, Sinatra/Rails) and JRuby + Hadoop/Cascading or Hadoop streaming for scale, trains a predict user overlap model (libsvm, statsample, NMatrix or R via RSRuby) and ships results to a Rails dashboard with D3; cleaning takes ~90% of the effort and the pipeline proves practical.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:12Sharethrough
- 1:40Outline
- 5:34Visualizing Friendships
- 10:37Phase 2
- 11:12Start With This
- 11:58Where does your data come from?
- 14:17Tracking Sandy
- 17:29Finding Bad Data
- 21:28Cascading JRuby
- 23:22Phase 3
- 27:38Phase 4
- 29:12Personal Example
- 30:28Ruby Improvements
