Big Data in Memory
summary
Claims a binary, model-driven encoding cuts memory and CPU drastically while keeping the same Spring APIs and workflow. Uses Spring Integration, Reactor, GemFire and Java NIO to replace fat Java objects with generated binary codecs; demo shows 7.4kB XML → 25kB Java → 450 bytes per record and running 10 million derivative contracts in memory on a laptop. Benchmarks: 5M trades: standard Java serialization 200s vs binary nio 2s, much smaller heap, far fewer GC pauses, and RTI Telco pipeline handling ~100k/s average (peaks near 800k–10M/s).
AI-written summary · about
chapters (9)
- 0:00Introduction to big data
- 1:53Memory efficiency in Java
- 9:36Demo setup and trade data
- 22:18Serialization performance tests
- 30:04Binary data advantages
- 41:21Real-time processing at scale
- 53:21Telco network analytics
- 1:00:03Optimization strategies
- 1:05:19Q&A session
related talks
51mIn-Memory Computing - The Big Picture
43mCaching and Beyond - Smarter Data Processing with Java
48mIn-Memory-Computing - The Big Picture
16mHeapBuffers: Why not just using a binary serialization format for your managed memory?
45mCaching Unleashed: Supercharge Your Database Application
39mMemory Efficient Java
33mConversational Big Data
40mLudicrous Speed: Designing For Performance on the JVM