Streaming video analysis in Rust using Pravega
summary
Case study of a scalable real‑time video analysis pipeline written in Rust using GStreamer for frame processing and Pravega for atomic, low‑latency streaming to object storage; an NVIDIA GPU object detector outputs JSON, Flink runs SQL‑style queries, and a web UI navigates multiple streams. Tests show Pravega sustains throughput as writer count grows while Pulsar and Kafka degrade. The team builds a client library with FFI to expose the same stateful client across languages and the end‑to‑end demo operates successfully.
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:32Gstreamer
- 3:53Benefits of a streaming architecture
- 5:17Pravega design
- 6:40End to end pipeline
- 7:09Demo
- 8:39Software Values
- 9:24Rust's values
- 9:47The usual narrative
- 12:26Our problem
- 13:31Client design
- 14:58What's in a method signature?
- 18:06Thinking about ownership
- 20:27Rust doesn't need the visitor pattern
- 22:49Using the type system to enforce invariants - 3
- 24:34Problems we faced
- 25:17The reactor pattern
- 25:56The async reactor pattern
- 27:43What we came to love
- 29:26Tools for error handling, tracing, and debugging
related talks
29mLessons Learned: Porting a Streaming Data Pipeline from Scala to Rust.
26mThe art of programmatic videos with Rust
16mEmpowering Fearless Data Engineering
30mGStreamer & Rust – A perfect match
40mStreamling: Lightweight, Extensible Streaming on DataFusion
46mAdopting Rust by gracefully oxidising web applications
38mTokio, Rayon, and the GPU: Profiling a Radiology AI Pipeline in Rust
20mMastering Rust on Edge and Cloud — AI/IoT Case Studies