From Bytes to Brilliance : Optimizing Large File Delivery in REST APIs
summary
A Spring Boot REST API that initially loads entire zipped S3 objects into a byte[] causes OOM at ~1 GB and crashes under concurrent downloads; switching to StreamingResponseBody and streaming chunks with Deflater.BEST_COMPRESSION fixes memory spikes and shrinks heap use from ~5 GB to ~3 GB, but corrupted ZIPs appear until output streams are properly closed. Also align HTTP timeouts—use Tomcat idle timeout 3m and Spring async timeout 2m in this case—to avoid 503s and half-written downloads.
AI-written summary · about
chapters (13)
- 0:00Introduction and context
- 1:34Speaker introduction
- 2:36Problem with REST for large files
- 4:24The memory bomb mistake
- 7:35Streaming as the solution
- 8:58Fixing corrupted zip files
- 10:00Adding compression
- 12:08HTTP headers for streaming
- 13:51Resource usage improvements
- 14:13Timeout alignment challenges
- 16:41Final timeout configuration
- 17:24Key takeaways and pitfalls
- 19:10Conclusion and Q&A
related talks
47mSpring REST Gigantomania: Handling Multi-Gigabyte POST Requests
36mThe Perils of Large Files
48mLean Spring Boot Applications for the Cloud
20mTuning Flink Filesystem Plugins for State Persistence
52mLean Spring Boot Applications for the Cloud
49mBuilding Lean Spring Boot Applications for the Cloud
51mIn-Memory Computing - The Big Picture
23mDecoding Javascript: Loading Performance