The Wikimedia infrastructure
summary
Wikimedia runs its own mid‑size physical network of about a thousand servers across Ashburn and Dallas with caching POPs in Amsterdam and San Francisco and serves 500 million uniques/month and ~190,000 HTTP reqs/s. The stack uses gnsd for geodns, ipvs/LVS and Nginx, Varnish with a custom director, Puppet (~60k LOC), MariaDB sharded into seven clusters, OpenStack Swift for ~800 TB and 300+ million files, and Elasticsearch; custom tools like Pall and Parsoid enable automated failover and deployment and are reported to work reliably.
AI-written summary · about
chapters (12)
- 0:00Intro
- 2:57Open-source, freedom, community & transparency
- 4:07Limited resources
- 4:41Physical topology
- 6:28Network architecture
- 7:21System architecture
- 8:18Load balancing: layer 2/3
- 10:04Load balancing (& caching): layer 7
- 12:39Main appserver stack (cont'd)
- 13:24Internal services
- 14:09Media storage infrastructure
- 15:09Wikimedia Labs
related talks
16mThe Wikimedia infrastructure
37mWikimedia Content API: A Cassandra Use-case
40mWikimedia Content API: A Cassandra Use-case #bbuzz
35mHow to Host a (Very) Popular Website for 30 Altairian...
34mFinding the Capacity to Grieve Once More
36mTesting Encyclopedias in Production
27mpostgresql.org: The hidden parts
38mThe SmugMug Tale