Achieving High Load in Advertising Technology
summary
Adform handles roughly 1.2 million bid requests per second with a tight 50–70 ms bid window, bidding on about 32% and winning under 1% of auctions. an Aerospike cookie store holds ~8 billion IDs at ~900k reads/sec and ~100k writes/sec with sub‑millisecond latency after replacing Cassandra and shrinking a cluster from many dozens to about eight nodes. swapping .net dictionaries for a fixed byte array binary search cuts bidder latency ~67% and yields ~2.5× capacity. tools include Kafka/Confluent, Spark, Vertica, Docker/Kubernetes, and .NET Core.
AI-written summary · about
chapters (28)
- 0:00Introduction and agenda
- 0:50Origins of Adform
- 2:14High load explained with Shinagawa station
- 3:00Latency throughput availability
- 5:55Real-time bidding in ad tech
- 9:52Bid request processing scale
- 12:20Cookie store and Aerospike
- 17:19Optimizing bid decision latency
- 20:12Data management platform overview
- 23:26Segment construction and data growth
- 24:31Lambda and Kappa architectures
- 27:25DevOps and private cloud
- 32:16OpenStack and multi-region setup
- 34:46Dotnet core migration benefits
- 39:05HDFS and Hadoop data storage
- 42:03Cost management for data storage
- 43:06Hive vs Spark performance
- 45:18Column stores and Vertica
- 46:22Aerospike for row storage
- 47:51Kafka and data integration
- 51:04Parquet compression and Spark
- 51:46GPU processing in databases
- 52:48Deep learning and cross-device tracking
- 53:30Persistent RAM technology
- 56:03Open source vs enterprise software
- 58:33CI/CD and deployment strategies
- 1:00:07Team culture and best practices
- 1:02:05Q&A
related talks
48m10 Billion a Day,100 Milliseconds Per: Monitoring Real Time Bidding
34mHow We Handle Over 1 Trillion HTTP Requests ...
28mMerging modern high-scale AdTech/MarTech at the edge
44mAdvertising at the Speed of Light (DMP Platform)
28mWhen Five Nines Is Not Enough
03mInterview
32mReal time Bidding Caching Use Case On The Google Cloud Platform
40mMastering small chunks of data in high frequency ad-tech systems