Solving Analytical Problems using Apache Spark

on YouTube ↗52m

" summary

Claims Spark runs up to 100x faster than Hadoop on large datasets and is proven here by combining RDD processing with a directed acyclic graph for lazy, fault‑tolerant recompute. The demo ties Spark Streaming (microbatches) to Cassandra via the Spark Cassandra connector, shows a weather‑station pipeline and movie‑count examples, and uses Kafka/Kinesis for ingestion and DataStax for stable deployment. The end result is parallel in‑memory analytics that reprocess only failed partitions and produce low‑latency results at petabyte scale.

AI-written summary · about

" chapters (12)

1. 0:00Introduction to Apache Spark
2. 3:29Big data and real-time analytics
3. 10:12Why use Apache Spark
4. 14:52Spark execution model
5. 21:12Resilient Distributed Datasets
6. 28:52Spark SQL and Cassandra
7. 34:50Batch vs streaming processing
8. 39:45Spark streaming architecture
9. 43:16Spark with Cassandra benefits
10. 46:45Lambda architecture overview
11. 48:33Kafka and Spark streaming
12. 51:01Optimization techniques

" related talks

• 40mSpark Analytics on Cassandra DataTim Berglund · Jfokus 2016
• 38mReal-time analytics with Apache Cassandra & Apache SparkChristopher Batey · Berlin Buzzwords 2015
• 59mStreaming Big Data w/Spark, Spark Streaming, Kafka, Cassandra & AkkaHelena Edelson · Philly ETE 2015
• 54mHarnessing the Power of Spark & Cassandra within your Spring AppSteve Pember · Spring I/O 2017
• 50mIntroduction to Real-Time Big Data with Apache SparkTaras Matyashovsky · JavaDay Lviv 2015
• 45mFrom Big Data to Fast Data: Apache SparkStefano Baghino · Codemotion Milan 2015
• 01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live codingOleg Klimenko · Java ON 2022
• 42mBuilding Robust Streaming Data Pipelines with Apache SparkZak Hassan · Open Source Summit North America 2017