Harnessing Power: Inside Sofascore’s Data-Driven Infrastructure
summary
SofaScore runs a hybrid pipeline using Firebase -> BigQuery + NATS -> ClickHouse with JetStream, Kubernetes and Amazon S3 backups, claiming support for ~25M monthly users and over 1.5 PB stored (about 1 TB/day, ~1–1.5 trillion rows). The system processes up to 2.5 billion messages/day and achieves sub-5s ingestion into ClickHouse, uses an in-house aggregator to cut query size by ~2,000×, and picks NATS over Kafka for lower TCO and sufficient throughput — the stack is deployed and operating in production.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:16SofaScore data infrastructure
- 8:27Real-time processing importance
- 9:33Legacy pipeline challenges
- 13:51Introducing NATS
- 15:12NATS and ClickHouse
- 18:26NATS 2.5B messages daily
- 20:55ClickHouse analytics speed
- 22:21Handling 2PB data
- 23:02Real-time analytics & personalization
- 24:48Personalization pipeline
- 26:58Future scaling plans
- 27:42Key takeaways
- 28:08Q&A
related talks
47mScaling: from 0 to 25 million users
17mUsing ClickHouse® Database to Power an Analytics and Customer Engagement Platform
28mMoving Towards Real-Time OLAP - With First Principle Thinking and Guided by Access Patterns
41mEnhancing Event Analysis at Scale: Leveraging Tracking Data in Sports
22mIt’s all about the db - a journey to accommodate 100k events per second
05mClickhouse & Goldsky: CryptoHouse
28mColumnstore Revolution
27mFast DIY O11y with OpenSrc Analytics & Visualization