Building a ChatGPT Data Pipeline with RisingWave Stream Processor and Cassandra Vector Search
summary
Shows a working pipeline that ingests Kafka Twitter feeds into RisingWave stream processor, runs SQL transforms and incremental materialized views, chunks text and computes 1,536‑dim embeddings, and writes vectors into Astra/Cassandra vector search using ANN indexes (JVector / DISN) for fast similarity. Architecture includes downstream storage, independent compute/storage scaling, and prompt construction for retrieval-augmented generation; demo cluster and screenshots exist and a free Astra tier ($25/month renewed) is offered. Twitter→RisingWave→Astra pipeline
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:03Talk agenda
- 3:32ChatGPT and generative AI
- 10:15RisingWave stream processor
- 14:08RisingWave architecture
- 20:53Vector databases in GenAI
- 27:20ChatGPT pipeline with RisingWave
- 30:10Resources and closing
related talks
49mBuild ChatGPT RAG Data Pipeline with RisingWave Stream Processor & Vector Store
27mBuild a ChatGPT RAG Data Pipeline
50mBuild a ChatGPT RAG Data Pipeline
29mFrom Quick Wins to Revolutionising Productivity & CX with GenAI
51mEvent Driven AI: Supercharging ChatGPT with RAG & LangStream
01h 01mStream Data Processing for Fun and Profit
57mEnter The Brave New World of GenAI with Vector Search
29mEnhancing Vector Databases with Real-time Stream Processing