Solving Classical Data Analytic Task by Using Modern Distributed Databases
summary
Shows how to replace slow Hadoop batch jobs with a pipeline that reads and writes Cassandra via the Cassandra Spark connector, using token-range-aware reads for parallelism and data locality and predicate pushdown so only needed rows hit memory. Demonstrates Spark batch ML (Naive Bayes on Iris), saving models as files/PMML for production, and micro-batch streaming (1s windows) that writes per-second aggregates back to Cassandra. Notes limits: some ML steps collapse to one node and GraphX is still alpha; examples run end-to-end.
AI-written summary · about
chapters (21)
- 0:00Start
- 1:17Agenda: Lambda Architecture
- 2:53Apache Cassandra
- 3:28Cassandra Architecture Overview
- 4:08Cassandra Query Language
- 5:04Apache Spar
- 9:07API
- 11:02Operator Graph: Optimization and Fault Tolerance
- 12:26Why Spark on Cassandra?
- 13:52How to Spark on Cassandra?
- 16:40Connecting to Cassandra
- 17:51Accessing Data
- 18:47Saving Data
- 19:53Type Mapping
- 20:22Mapping Rows to Objects
- 22:04Server Side Data Selection
- 31:52Spark SQL Example
- 33:14Spark Streaming
- 35:02Streaming Example
- 43:34Spark MLlib Example
- 51:24Questions?
related talks
40mSpark Analytics on Cassandra Data
21mLightning fast cluster computing with Cassandra and Spark
31mApache Cassandra Analytics: A Recipe to Move Petabytes of Data
26mUnleash the Power of Apache Cassandra: Scalable Recommendation Systems, Real-time Fraud Detection
32mMigrating Workloads from Mainframes to Cassandra - Problems, Experience and Learnings
35mThe What and Why of Cassandra Datamodelling at Scale
27mDesigning Modern Data Platforms with Apache Cassandra and Beyond
01h 48mBig Data made easy with a Spark (3.0)