How We Counted Traffic on Vertica

on YouTube ↗47m

" summary

Describes how Avito measured web traffic and computed detailed analytics (unique visitors and sessions) from hundreds of millions of events per day using Vertica. Explains the problem that unique counts are not additive across hierarchical dimensions (geography, product catalog) and presents the techniques and approach used in Vertica to produce accurate, highly detailed, ad-hoc reports.

AI-written summary · about

" chapters (18)

1. 0:00Introduction
2. 1:34About Avito and data scale
3. 3:02Vertica storage infrastructure
4. 6:16Data processing and normalization
5. 10:15Data structure and analytics
6. 14:13Cluster hardware
7. 15:39Data loading infrastructure
8. 16:41Installation Q&A
9. 21:04Vertica vs other systems
10. 23:34Cost and licensing
11. 24:596NF advantages
12. 33:51Traffic calculation task
13. 38:09Unique visitor counting algorithm
14. 41:34User action data collection
15. 43:41Time zone data processing
16. 44:23Data loading and updates
17. 45:07Analysts using the system
18. 45:49Data storage and deletion

" related talks

• 01h 03mVertica – High Performance Over Varying TerrainStephen Walkauskas · Vaccination Database Talks (Second Dose) 2021
• 20mVertica & Column StoresAndy Palmer · A Celebration of Mike Stonebraker (April 12, 2014)
• 04mIgor Marchenko from HP Vertica interviewed at Strata Rx 2013Igor Marchenko · Strata Rx Conference 2013
• 37mMeasuring 2.0 – Building the next generation webanalytics ...Niels Basjes · Berlin Buzzwords 2016
• 47mRethinking Relational Database CRUD Design Patterns Since We Live in a Big Data WorldChris Bohn · HighLoad++ 2017
• 07mJoshua Bixby interviewed at Velocity 2010Joshua Bixby · Velocity 2010
• 27mTales at Scale: Analytics at 1000 QPS and BeyondGian Merlino · Open Source Summit Latin America 2022
• 44mNear-Real-Time Event Analytics in a High-Load ProjectAleksandr Krasheninnikov · HighLoad++ 2015