Building a Personalised Home Feed #bbuzz
summary
eBay Kleinanzeigen builds an on‑demand personalized home feed that scores and mixes results from a 28 million‑item Elasticsearch index using user features derived in near‑real‑time from event streams. User interactions and searches land in Kafka Streams pipelines and are persisted to compacted Kafka topics (serving as the source of truth) with local lookup via RocksDB for low‑latency reads. The system serves thousands of feeds per second, handles short‑lived inventory by avoiding materialized per‑user feeds, and flags the Elasticsearch cluster as the primary scaling concern.
AI-written summary · about
chapters (5)
related talks
20mDataEngBytes 2023 - PER-T1-06 - Kamal Hossain
34mBuilding a WebScale Search Engine with a Data Driven Architecture
36mReal Time Big Data Analytics with Kafka, Storm, HBase #bbuzz
28mA complete Tweet index on Apache Lucene #bbuzz
42mUsing Data Orchestration for speed, reliability and real-time content
22mGenerating a Billion Personal News Feeds
21mReporting Systems Built on Top of Hadoop & Lucene
32mBusiness Intelligence in Microservice architecture #bbuzz