Zero-Change Object Transmission for Distributed Big Data Analytics
summary
Zero-Change Object Transmission (ZCOT) copies live Java objects into a shared exchange space mapped at the same addresses across JVMs and exposes a chunked metadata server that manages 256MB chunks, chunk maps and RPCs; receivers trigger page-fault pulls to fetch remote chunks, GC hooks release chunks, and a front-end metadata stream preserves compatibility with existing serialize/deserialize APIs. Prototype integrates with Spark and Flink, runs on a 4-node 100Gbps cluster, yields microbenchmarks ~2.28x, serialization-phase speedups up to 4.19x, and up to 23.6% application runtime improvement (13.9% average in Spark).
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:28Distributed Big-data Analytics
- 1:12Workflow of Big-data Processing
- 2:32Costly Inter-JVM Communications
- 4:36Existing S/D Optimizations
- 6:44How to Achieve This?
- 7:51Challenges for ZCOT
- 8:17Space Construction: DCDS
- 9:13Compatibility with Applications
- 10:55Distributed Memory Management
- 11:57RPC Interfaces
- 13:00The Workflow of ZCOT
- 15:57Experimental Setup
- 16:19Mircobenchmark
- 18:06Flink Performance
- 18:36Conclusion
related talks
24mLiving on Channel ZGC
51mHigh Bandwidth Data & Low-latency Java
42mBuilding Bulletproof Concurrent Applications on the JVM
52mDistributed Object Graphs with MicroStream
15mFaster Data Access with MicroStream
44mJava at Speed: Building a Better JVM.
46mZGC: A Scalable Low-Latency Garbage Collector
21mFaster Data With Faster Java