Radanalytics.io streaming applications workshop

on YouTube ↗01h 47m

" summary

Shows a concrete pipeline: synthetic social updates (Jane Austen + reviews) stream into Kafka, Apache Spark Structured Streaming applies a spaCy + VADER sentence-level sentiment UDF, and results write back to a second Kafka topic. Notebook demos report ~90 messages in 10 seconds, ~17k updates total, top user posts 28 times, ~3,563 distinct users. Deployment uses Kafka + Structured Streaming on OpenShift/Kubernetes with a small Spark cluster and a microservice visualizer; the transformer service runs and the dashboard surfaces most‑positive and most‑negative examples.

AI-written summary · about

" chapters (13)

1. 0:00Start
2. 0:51Forecast
3. 4:27Multitenant compute clusters
4. 5:36One cluster per application
5. 15:14Apache Spark
6. 19:41Example: word count
7. 22:41Streaming data
8. 23:44Structured queries
9. 25:02Structured query in Spark
10. 25:39Query planning
11. 26:25A naive plan
12. 26:57An optimized plan
13. 28:10radanalytics.io

" related talks

• 01h 02mBig Data Analytics in Near-Real-Time with Apache Kafka StreamsAllen Underwood · NDC London 2020
• 34mActionable Insights with Real-time Streaming Analytics of Customer ReviewsDaniel Wrigley · Big Data Conference Europe 2019
• 45mReal-time, Streaming Advanced Analytics....Chris Fregly · Data By the Bay 2016
• 01h 25mApache Spark for Big Data ProcessingLudwine Probst, Ilayaperumal Gopinathan · SpringOne2GX 2015
• 43mSentiment Analysis in Action: Building Your Real-time PipelineOlena Kutsenko · Big Data & RPA Conference Europe 2024
• 30mDeploy end to end ML pipeline using Apache spark ...Nick Pentreath, Prashant Sharma · Scale By The Bay 2019
• 30mSmart Streams: A Real-time framework for scoring Big and Fast DataVedant Jain · PyData Miami 2019
• 01h 26mData Driven Action: a primer on Data ScienceSarah Aerni, Srivatsan Ramanujam, Jarrod Vawdrey · SpringOne2GX 2015