Near Real Time Processing of Time Series Data with HBase
summary
Shows how HBase time‑series writes use a salt prefix to avoid hot regions but break range scans, and how implementing MultiSegmentTableInputFormat lets a single MapReduce job consume multiple Scan objects so rows remain ordered by timestamp. Replaces the old pattern of launching a job per prefix, keeps writes distributed across region servers, and yields a single ordered, near‑real‑time MapReduce pass over the data with measurable performance gains.
AI-written summary · about
chapters (8)
- 0:00Introduction to real-time processing
- 1:57Problem with customer journey analysis
- 4:04HBase cluster design challenges
- 6:42Monitoring and metrics setup
- 7:28MapReduce job limitations
- 8:38Solving hotspots with prefix design
- 10:08Use case for YouTube space processing
- 11:04Handling large single MapReduce jobs
related talks
45mFrom Big Data to Big Information
55mApache HBase: crazy dances on the elephant back
29mBig Data Analysis
59mHadoop Essentials for the SQL Server Professional
45mSolving Real & Big (Data) Problems Using Hadoop
53mBig Data: from mammoth to elephant
42mA Tale of One Billion Time Series
07mTime Series Databases