Stream Algorithms for Big Data Processing Tasks

on YouTube ↗23m

" summary

Describes using streaming (stream-processing) algorithms to analyze terabytes of user-behavior data for ad targeting in a DMP. Covers techniques to quickly estimate large-scale statistics and distribution shapes of various features using stream algorithms.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 2:33Stream algorithms for ads
3. 8:16Traffic interest analysis
4. 10:44Median algorithms
5. 15:09Greenwald-Khanna algorithm
6. 16:36Spark implementation
7. 17:44Q&A

" related talks

• 27mStrike gold (and avoid drowning) with streaming algorithmsGuy Smoilovsky · Reversim Summit 2019
• 32mBig Data Processing : Where do we go next ?Roshan Naik · Flink Forward Global 2021
• 45mData Stream Processing Concepts and ImplementationsMatthias Niehoff · Devoxx Belgium 2017
• 43mSegmenting 600 Million UsersArtem Marinov · HighLoad++ 2016
• 38mThe story of building a real-time streaming analytics systemRenat Idrisov · Backend Conf 2018
• 44mReal-time Data Processing with Apache Spark Structured StreamingEiti Kimura · QCon São Paulo 2019
• 47mOne streaming to rule them all: Streaming as the foundation of an analytics ecosystemDmitriy Shirokov · HighLoad++ 2024
• 39mHigh-volume Data Streaming and StorageGleicon Moraes · QCon São Paulo 2019