When to Use Native Extensions in Rust: Accessible, Safe Performance
summary
Use native Rust extensions when profiling shows the critical path still dominates after optimizations and concurrency testing. Rust gives near-C/C++ performance plus memory and concurrency safety, supports incremental integration without full rewrites, and a concrete example is Pants moving orchestration, caching and management of hundreds–thousands of threads/processes into a Rust extension (five years in production) that dramatically reduced latency according to profiles and flamegraphs.
AI-written summary · about
chapters (31)
- 0:00Start
- 0:30Presentation plan
- 0:50Who am I
- 2:00Pants - how we use Rust + Python
- 3:22Python is usually slow
- 3:47But it’s not always a problem
- 6:02Profiling
- 6:35Flame graphs
- 6:51GIL profiles
- 7:19Some options when there’s a performance problem
- 7:32Optimizing
- 8:20Python’s concurrency
- 10:06Option 3
- 10:59Rewriting everything in another language
- 11:51Native extensions: a mix of Python and another language
- 12:16Extension - demo
- 12:46When native extensions can be worth it
- 13:45Why Pants chose a native extension
- 14:37Native extensions add complexity
- 15:16Some options for native extensions
- 16:17Rust can use Python too!
- 16:30C and C++ don’t have memory safety.
- 17:06Memory vulnerabilities are common
- 17:33Rust - memory safety without a garbage collector. The compiler checks memory safety via its borrow checker.
- 17:59Rust - fearless concurrency
- 18:17Rust - performance parity
- 18:33Rust - other advantages
- 19:08Rust - some disadvantages
- 19:39Resources on Rust
- 20:10Summary
- 20:59Follow the Pants team’s conversation
related talks
53mRust: Reach Further!
01h 04mRust-ifying Your C# Codebase: A Tale of Adventure and Transformation
50mExtreme Performance, Safe Execution and Elegant Code with Rust
40mTurbo Rails with Rust
47mRewriting a Ruby C Extension in Rust
56mWhen to Choose Rust
40mNine Rules for Writing Python Extensions in Rust
21mExtending the horizons of R with Rust