Data driven matchmaking streaming at HyperConnect
summary
HyperConnect rebuilds its 1:1 video matchmaking to run on Apache Flink, handling peaks of 5,000 req/s that translate to about 12 million candidate comparisons per second and 180 million daily discoveries. The system uses Kafka or a custom HTTP source, Redis for shared state, Kubernetes with blue‑green deploys, a block pairwise MapReduce to cut N^2 costs, iterative streams and application-level backpressure to keep cycles under one second, and a streaming multi-armed-bandit for personalization.
AI-written summary · about
chapters (5)
- 0:00Introduction to redesign
- 3:56Matchmaking architecture overview
- 15:26Infrastructure and deployment
- 17:32Stateful matchmaking system
- 22:53Summary and conclusion
related talks
15mOn-demand low-latency feature generation at Hyperconnect
18mThe role Stream Processing plays for an Opti Channel Experience
25mData: Moving with the flow
21mReal-time Stream Analytics and Scoring Using Apache Flink, Druid & Cassandra
43mUsing Queryable State to update ML Algorithms per datapoint for...
27mData enrichment patterns with Apache Flink
39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
15mData science at scale using Apache Flink: Dynamic model serving and real-time feature generation