Working with Spark and YugabyteDB

on YouTube ↗31m

" summary

Overview of using Spark with YugabyteDB using the YugabyteDB Spark connector to run Spark shell jobs against a Yugabyte Cloud cluster on cloud.yugabyte.com with CA cert authentication. Demonstrates creating tables (emp with ~10–15 rows, emp_right) and a books table with JSONB native support, computing per-job ranks via a Spark window function (partition by job order by salary) and writing results back, showing predicate pushdown, cluster/topology awareness, and an upsert that adds an authors element to id 6.

AI-written summary · about

" chapters (15)

1. 0:00Session introduction
2. 1:48YugabyteDB architecture
3. 4:29YSQL features
4. 8:24Distributed transactions
5. 9:13JSON support in YugabyteDB
6. 10:42Spark connector
7. 12:29Connector optimizations
8. 13:10YugabyteDB Cloud setup
9. 16:45Demo table creation
10. 19:15Spark connection
11. 21:35Reading data from YB
12. 23:21Writing data to YB
13. 26:44JSON in Spark
14. 28:38JSON updates
15. 30:09Closing remarks & Q&A

" related talks

• 27mSparking our way to YugabyteDB: Our journey from ScyllaDB toPrashant Gajera · Distributed SQL Summit Mumbai 2026
• 26mEfforts Toward Adoption and Uncovering Hidden FeaturesTatsuhito Kasahara · Distributed SQL Summit Asia 2024
• 01h 29mYugabyteDB DBA FundamentalsMark Kim · Distributed SQL Summit 2022
• 30mBuilding Scalable Spring Applications with YugabyteDBTakeshi Ogawa · Distributed SQL Summit Asia 2023
• 01h 27mYugabyteDB YSQL DevelopmentSeth Luersen · Distributed SQL Summit 2022
• 01h 21mYugabyteDB YCQL DevelopmentMark Kim · Distributed SQL Summit 2022
• 30mBuilding Cloud Native Applications on Distributed SQL PlatformSHIGERU URUSHIBARA · Distributed SQL Summit Asia 2023
• 02h 27mDistributed SQL Tuning and Optimization in YugabyteDBFranck Pachot · Distributed SQL Summit 2022