Using Tokio for CPU-Bound Tasks (Works Really Well)
summary
Claim: Tokio as runtime for CPU-bound tasks actually works for analytics engines because DataFusion models operators as async streams and processes record batches in-task, enabling core-local execution and vectorized work per task. Implementation uses a separate Tokio runtime for CPU work, integrates with async IO, and benchmarks approach parity with DuckDB on core scaling. Gotchas include mixing IO and CPU on same runtime throttles network and the need to yield based on the runtime task budget, fixed upstream.
AI-written summary · about
chapters (11)
- 0:00Tokio intro and CPU tasks
- 3:20Why Tokio suits CPU work
- 5:51Tokio misconceptions
- 8:19Tokio task overhead
- 10:03Data flow graphs and streams
- 12:50Keeping data on same core
- 14:13Multiple runtimes for CPU and IO
- 16:19Real-world performance
- 18:02Debugging thread contention
- 20:27Task cancellation in Tokio
- 21:52Conclusion
related talks
05m“Using async-profiler to Find Tail Latency Issues”
38mTokio, Rayon, and the GPU: Profiling a Radiology AI Pipeline in Rust
45mKeynote | Actors with Tokio – a lesson in ownership
16mEmpowering Fearless Data Engineering
29mI/O in Rust: the whole story
32mTechniques for writing concurrent applications with asynchronous I/O
51mThousands of Async Tasks per Second in the Cloud on Rust/Axum/Tokio
06mZero-copy Transfers with io_uring and Rust