How We Counted Traffic in Vertica
summary
This talk describes how Avito implemented traffic analytics (unique visitors and sessions) on HP Vertica. It explains the challenge that unique counts are non-additive across hierarchical dimensions (like geography or product categories) and how they built a MapReduce-style solution on top of Vertica’s SQL engine to compute detailed, accurate, ad-hoc reports over tens of billions of rows.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:34Avito’s scale and challenges
- 3:24Vertica storage architecture
- 5:44Event stream processing
- 8:12Handling massive events
- 10:45Analytical capabilities and setup
- 14:13Query performance optimizations
- 16:41Q&A on installation and backups
- 18:34Vertica advantages and use cases
- 21:48Cost efficiency and licensing
- 23:34Pricing and 6th normal form
- 26:27Narrow vs wide tables
- 29:35Partitioning and management
- 31:41Data loading and updates
- 33:09Analyst workflows
- 33:51Traffic calculation challenges
- 37:22Non-additivity in metrics
- 39:17Solving aggregation issues
- 41:34Data collection and pipeline
- 44:23Q&A on sources and storage
related talks
50mOur experience at Etsy com with building “Big Data” analytic capabilitie
35mUnifying Analytics
58mAvito Data Store: Analytics for a Microservices Architecture
56mSpark-driven Audience counting
30mAn architecture for big data: Supporting journalism through transparency in advertising
09mapproach to data
01h 01mUsing Power View and Hadoop to Unlock Hidden Markets
27mTales at Scale: Analytics at 1000 QPS and Beyond