~/talk

Hydroflow: A Compiler Target for Fast, Correct Distributed Programs

on YouTube 01h 02m
Talkat OOPSLA 2023 · 2023-10-23..10-27· [distributed-systems, compilers-and-language-design]

speaker

name:Joseph M. Hellerstein
links:none on file

resources

video:https://youtu.be/YpMKUQKlak0
slides:none on file

summary

Hydroflow presents a Rust-based dataflow IR that compiles into networked transducer binaries, uses a semilattice formalism to make CRDT-style state composable, and provides three concrete transformations: automated refactoring into deployable blocks, safe replication via lattice-based types, and key-based partitioning for sharding. Benchmarks match a handwritten C++ key‑value store (Anna) for latency and scaling. The system uses monotonicity/Calm analysis to avoid coordination where possible and offers provably correct rewrite rules to optimize protocols like Paxos, though cost‑based automation remains work in progress.

AI-written summary · about

chapters (20)

  1. 0:00Introduction and speaker welcome
  2. 3:14Motivation for programming the cloud
  3. 8:54Key value store example Anna benchmark
  4. 13:27Hydroflow compiler and transducer IR
  5. 15:11Transducer IR and data flow programming
  6. 16:57Building a key value store in Hydroflow
  7. 20:29Replicating the key value store
  8. 21:55Advanced Hydroflow operators and semantics
  9. 22:18Compilation and performance optimization
  10. 24:42Matching handwritten C++ performance
  11. 25:47Consistency and the CALM theorem
  12. 31:02Semilattices and monotonicity
  13. 33:28Reading without coordination
  14. 36:16Coordination protocols in Hydroflow
  15. 37:42Lattice composition and correctness
  16. 38:03Data flow types and monotonicity analysis
  17. 41:55Optimizations across transducers
  18. 43:39Optimizing Paxos with Hydroflow
  19. 48:10Future directions and open challenges
  20. 50:14Talk conclusion and Q&A introduction

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.