The Wikimedia infrastructure
summary
Wikipedia serves about 500 million unique users and 20 billion monthly page views peak at ~190,000 HTTP requests per second and runs ~1,000 servers across two primary DCs (Ashburn and Dallas) plus caching POPs. The stack uses gnsd geodns for DC mapping, ipvs/LVS and nginx with chained varnish caches, MariaDB shards for writes and ElasticSearch for search, OpenStack Swift for ~800 TB and 300M+ files, Puppet and Salt for automation, Parsoid and small Node services, and a custom Pall BGP failover that proves effective.
AI-written summary · about
chapters (12)
- 0:00Intro
- 2:57Open-source, freedom, community & transparency
- 4:07Limited resources
- 4:41Physical topology
- 6:28Network architecture
- 7:21System architecture
- 8:18Load balancing: layer 2/3
- 10:04Load balancing (& caching): layer 7
- 12:39Main appserver stack (cont'd)
- 13:24Internal services
- 14:09Media storage infrastructure
- 15:09Wikimedia Labs
related talks
16mThe Wikimedia infrastructure
35mHow to Host a (Very) Popular Website for 30 Altairian...
34mFinding the Capacity to Grieve Once More
43mTools and Team for 20 Billion PageView/Month
56mKeynote Interview
36mTesting Encyclopedias in Production
47mScaling: from 0 to 25 million users
27mpostgresql.org: The hidden parts