Real Time Machine Learning Visualization with Spark
summary
Adds callback hooks to Spark ML algorithms (K‑Means, logistic and linear regression) to emit iteration-specific messages including centroids, cost and a sampled prediction of cluster membership; uses Akka actors for a driver ↔ client task channel and a simple task observer to accept stop commands; streams updates to the browser via HTML5 SSE (Spray chunked responses) and renders with Plotly/D3; demo runs on a ~10GB sample, 20 max iterations, and successfully lets training be stopped when cost flattens.
AI-written summary · about
chapters (6)
- 0:00Introduction to real-time ML
- 1:58Simple ML algorithms overview
- 3:49Why visualize ML training
- 4:33Live demo of visualization
- 8:13How real-time visualization works
- 16:45Summary and Q&A
related talks
17mMachine Learning at the Speed of Light with Spark
39mMachine Learning with Scala on Spark
51mApache Spark and machine learning: dreams and realities
40mReal-time Learning with Spark
26mOptimal Approaches for Real-Time Machine Learning with Apache Spark on Kubernetes.
50mApache Spark vs Machine Learning
51mSpotlight on Emerging Data Science Worskspaces and Spark as Development Levers
11mScaling R with Spark