JSON: push the limits

on YouTube ↗45m

" summary

Claims that processing huge JSON without holding the whole string reduces memory from gigabytes to tens of megabytes and removes main-thread jank. Uses Node/browser streams, TextDecoderStream and a custom chunked parser plus a streaming stringify implementation; avoids promises and intermediate buffers, counts bytes via a proxy ReadableStream for progress. On real workloads a 500 MB JSON drops from ~1 GB peak to ~30–112 MB and becomes processable in browsers and webpack; the streaming approach is implemented and shipped. streaming stringify chunked JSON parse

AI-written summary · about

" chapters (19)

1. 0:00Introduction
2. 1:03Large JSON challenges
3. 3:55Defining large JSON
4. 7:09JSON operations testing
5. 14:09Memory and time issues
6. 14:52Streaming JSON
7. 15:54JSON file streaming
8. 17:20Streaming optimization
9. 20:49Parsing JSON fragments
10. 22:35Fragment parsing implementation
11. 26:56JSON limitations summary
12. 29:05JSON loading progress bar
13. 32:16Memory optimization for loading
14. 35:27Loading progress calculation
15. 37:13Progress bar rendering issues
16. 39:21Binary JSON representations
17. 41:50Binary JSON prospects
18. 42:11Conclusion
19. 43:16Q&A

" related talks

• 37mUsing small data in the client instead of big data in the cloudAnton Dubrau · PyData Berlin 2016
• 51mFront End DataJing Chen, Jake Archibald, Nolan Lawson, Matt MacPherson, David Fahlander, Glynn Bird · Edge Conference 2015
• 27mScalable Data VisualizationZach Nation · React.js Conf 2015
• 29mGolang and JavaScript: The Future of CloudDaniel J Blueman · FOSSASIA Summit 2020
• 43mStorytelling in the Age of JavaScriptSarah Drasner · FITC Toronto 2017
• 37mVisualizing Persistent Data StructuresDann Toliver · Strange Loop 2014
• 28mJavaScript JournalismRay Daly · JSConf US 2013
• 48mHow to Analyze Terabytes of Data from GitHub Archive at High SpeedAliaksandr Valialkin · Big Data Conference Europe 2025