Scaling: from 0 to 25 million users
summary
SofaScore grows to 28 million monthly active users with 1.4 PB/month and 400 billion requests, peaking at 1.6M req/s, and scales by baking a multi-layered HTTP cache using varnish, sharding by URL, and running 50–100k purges per minute to reach a 99.5% cache hit; stack choices move from MongoDB (replication and locking issues) back to PostgreSQL, use ClickHouse for analytics, NATS for realtime pub/sub, Kubernetes and Longhorn on bare metal, and a hybrid failover to cloud — net result: 10x lower network costs and stable production at scale.
AI-written summary · about
chapters (20)
- 0:00Introduction to Sofascore
- 1:38Early scaling challenges
- 3:52Proactive caching strategies
- 5:11Surviving massive traffic
- 7:26Cloud migration and Varnish
- 10:24Database evolution
- 12:52Database migration lessons
- 14:45Asynchronous architecture
- 16:06Advanced caching techniques
- 17:35HTTP caching headers
- 19:48Moving to bare metal
- 22:39Infrastructure efficiency
- 25:22Traffic spikes and monitoring
- 26:50Performance optimization
- 28:44Global latency and edge routing
- 30:22Data center resilience
- 32:54Kubernetes on bare metal
- 35:14Real-time updates with NATS
- 37:12Analytics at scale
- 39:45Conclusion and Q&A
related talks
45mScaling: from 0 to 20 million users
30mHarnessing Power: Inside Sofascore’s Data-Driven Infrastructure
38mMoneyball Generation: Sports & Tech
46mScaling With Friends
24mScaling to Billions of Internet Observations per Day
26mHow We Prepped Our Infrastructure for a Super-Bowl Sized Traffic
21mUse cases that drive the need for NoSQL databases
20mLessons learned scaling our SaaS on Postgres to 8+ billion events