10x CouchDB Performance Gains for a AAA Game Launch
summary
Benchmarks using Apache Bench and iPerf reveal a 1 Gbps AWS networking cap and huge latency spikes for concurrent view queries. Changes: enable higher AWS networking, double MochiWeb acceptor pool from 16 to 32, increase Erlang internal buffers (60 MB message buffers and 4 MB socket buffers), and validate with curl --trace-time, ss, Recon and msacc. Result: DBs info header latency drops from ~100 ms to ~12 ms, response body from ~1.2 s to ~0.5–1 s, throughput rises from ~6k req/s to 25k sustained and up to 60k in some configs.
AI-written summary · about
chapters (24)
- 0:00Introduction and context
- 1:54Customer challenge and goals
- 4:24Benchmarking reality check
- 7:14How CouchDB sharding works
- 10:21Cluster request handling
- 13:11Network bottleneck discovery
- 14:35Benchmarking methodology
- 16:20Focusing on view requests
- 17:03Disk IO analysis
- 18:24Erlang process insights
- 20:29Measuring message queues
- 23:20CPU and RAM evaluation
- 23:41HTTP request tracing
- 25:29CouchDB layer cake
- 26:33Curl trace time analysis
- 29:02DBs info endpoint deep dive
- 30:25Config API bottleneck validation
- 30:47TCP backlog and acceptor pool
- 32:55Benchmarking best practices
- 34:17Testing TCP acceptor pool changes
- 35:21CPU scheduler analysis
- 38:12Erlang network tuning
- 39:59Final performance results
- 41:47Key learnings summary
related talks
55mCouchDB to the Edge
23mArchitecting for Performance, Scalability and Real-time Data with AWS
47mScaling: from 0 to 25 million users
01h 03mExtreme Performance Architecture
26mHow We Prepped Our Infrastructure for a Super-Bowl Sized Traffic
40mLatency, Throughput & Fault Tolerance: Building a Modern Database
47mScaling: from 0 to 25 million users
41mUsing Databases to Pull Your Application Weight