How Small Changes Can Make Huge Waves
summary
Braze runs at massive scale—11 billion user profiles, 8 billion Sidekiq jobs/day and 6 billion API calls/day—and uses feature flippers to roll changes from 1% to 100%. A frequency‑capping rewrite moves logic into MongoDB’s aggregation pipeline and cuts median job time from ~100ms to ~75ms (25% savings), eliminates huge network/RAM pulls and prevents OOMs. Content Cards moves per‑user card rows into Postgres inserts to handle million+ writes reliably. Deployments use small betas, DataDog tracing, and behavioral + performance gates.
AI-written summary · about
chapters (10)
- 0:00Start
- 2:51How Does It All Work?
- 11:57Percentage Rollout Example
- 13:05Frequency Cupping
- 16:59User Collection Example
- 19:35Frequency Capping Problems
- 24:21NEW Frequency Capping Algorithm
- 25:15Feature Flipped Deployment
- 28:44Write Behavior
- 34:19Lessons Learned
related talks
47mScaling: from 0 to 25 million users
33mData: The Land DevOps Forgot
25mScaling Data Platforms: Managing Growth from Zero to Billions of Transactions
23m"In the Belly of the Whale: Operations at Twitter"
35mSidekiq at Gusto: The Parts Where People’s Jaw Drops
32mZero Downtime Deploys Made Easy
28mHow to get away with refactoring
20mScaling up a Super-App