Analysing your CDN usage

on YouTube ↗21m

" summary

CDNlysis pulls gzip CloudFront logs from S3, parses changing headers, de-duplicates records and streams CloudFront logs to InfluxDB for real-time queries and Grafana dashboards. It's a Go binary (Docker option) that uses persistent queues and a producer/consumer pipeline, limited by Amazon to about 500 req/sec, and it processes 7M records in 4–5 hours. Output drives concrete actions: per-tenant bandwidth chargebacks, geo-based API moves, and choosing lower-bitrate encodes where delivery is slow.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 1:21Preface
3. 2:29Optimising Static content delivery
4. 3:21Pricing
5. 4:46What can rich analysis do?
6. 16:13State Flow

" related talks

• 53mCDNs 101: An Introduction to Content Delivery NetworksJake Ginnivan · NDC Oslo 2023
• 39mLarge scale streaming analyticsJay Palaniappan · Data Con LA 2019
• 32mWhat is a CDN & why Developers should Care about using oneArtur Bergman · GOTO Stockholm 2016
• 31mMigrating content distribution to Amazon CloudFront: Best practicesTed Middleton, Jibril Touzi · AWS Summit Online – Americas 2021
• 29mSex, Lies & Videotapes: The Truth Behind Your CDN LogsJavi B · RTC.On 2023
• 24mForensic Log Analysis with BigQueryAja Hammerly · MountainWest RubyConf 2015
• 55mCloud Analytics: from 0 to 100,000 events per second in 50'Didier Girard · DevFest Nantes 2016
• 31mLogging at Scale, Billion+ messages, 100K msg/secRohit Gupta · FOSSASIA Summit 2017