Getting Started with Spark
summary
Overview of Apache Spark basics with a live shell demo: downloads Spark Spark 2.2, opens the Spark shell, reads a local text file and reports 370,701 words, creates and composes RDDs, shows lazy evaluation (no work until a terminating action like count), inspects the web UI and DAG directed acyclic graph, starts master and workers, simulates a worker failure and observes task reallocation and recovery, and demonstrates filtering and a delayed task to show fault tolerance.
AI-written summary · about
chapters (18)
- 0:00Start
- 0:08Eclipse Collections
- 2:25Java Champions
- 4:55Distributed Processing
- 12:08Why Spark Was Invented
- 12:42What Is Apache Spark
- 13:20Download Spark
- 14:37What Is Lazy Evaluation and Eager Evaluation
- 14:50Eager Evaluation
- 16:10Lazy and Eager Evaluation
- 18:10Rdd
- 24:35Spark Shell
- 31:05Dag Visualization
- 40:44What the Spark Cluster Is
- 41:24How the Spark Cluster Works
- 44:05Invoke the Spark Class Master
- 45:04Start a Spark Shell
- 53:06Redundancy and Fault Tolerance
related talks
44m7 Tips to Get Started with Spark
50mIntroduction to Real-Time Big Data with Apache Spark
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
52mSolving Analytical Problems using Apache Spark
44mData Science at Scale with Apache Spark
53mLord of the Spark
01h 48mBig Data made easy with a Spark (3.0)
01h 25mApache Spark for Big Data Processing