User Attribution at Scale: All-Flash, ½ Trillion Objects
summary
Adjust migrates a Redis-based device store to Aerospike NVMe all‑flash, shrinking from ~40 Redis servers to 6 Aerospike nodes and cutting infrastructure cost by ~85%. Workload grows from ~40 req/min to ~750k req/s with ~160‑byte records and over half a trillion objects; Redis hits AOF fork stalls and single-thread limits, prompting a move. A namespace limit (4.2B) leads to a 10 namespaces workaround until Aerospike features like compression (0.6x), XDR and uniform balance stabilize performance at millions of ops/sec.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:58Who is Adjust?
- 1:43Who am I?
- 3:41Adjust and Attribution
- 5:06General Overview
- 6:53Redis small - 2012
- 8:14Redis BIG - 2013
- 9:20Redis Clustered - 2014
- 12:23Smoke in the machine room
- 18:02Migrating to Aerospike - 2015
- 19:09Using Aerospike - 2015
- 19:44Breaking Aerospike
- 22:15Scaling Aerospike
- 23:26Using Aerospike - 2021
related talks
38mAlternatives to Redis while not compromising on its speed
20mMaximizing the Benefit of Flash Storage for Aerospike
49mJourney with Aerospike: Architecture Best Practices
26mPowering Real-time Decisions With a Reduced Footprint
22mThe Art of the Possible
29mWhat breaks when Aerospike hits 6 million QPS
45mAerospike + Persistent Memory = Voila!
48mRebuilding a RT Datastore: the design, deployment & performance story