Empowering Fearless Data Engineering
summary
Claims that Rust matches C/C++ speed while preventing memory bugs, demonstrates rayon-based parallelism and tokio async for streaming, and shows a concrete log pipeline that processes gigabytes of logs daily using zero-copy handoff and custom data layouts. The demo divides files across all CPU cores, reduces memory overhead, and produces a fast, reliable log parser that scales without extra GC pressure. Examples include real-time analytics, stock-market ingestion, and IoT sensor streams.
AI-written summary · about
chapters (9)
- 0:00intro
- 0:26why choose rust?
- 1:08rust's performance and safety
- 1:56concurrency and productivity in rust
- 2:37advantages of rust
- 3:44common applications of rust
- 7:37rust in data engineering
- 12:42case study: real-time analytics with rust
- 13:56conclusion and future of rust
related talks
29mLessons Learned: Porting a Streaming Data Pipeline from Scala to Rust.
28mRust and Scala, Sitting in a Tree….
59mParallel Programming in Rust: Techniques for Blazing Speed
56mWhen to Choose Rust
53mRust: Reach Further!
40mRust’s approach of getting things right
40mWhy Rust?
39mChallenges of Building Blazing Fast Data APIs