Adventures at HyperScale!

on YouTube ↗53m

" summary

Have I Been Pwned handles ~17 billion requests a month, serves ~430 TB, and stores ~6 billion unique emails plus ~1 billion password hashes while being run by two people. Architecture uses Cloudflare workers at the edge, Cloudflare CDN with Cloudflare cache reserve to avoid origin egress, and Azure for compute and storage with Azure SQL Hyperscale for main data; a custom autoscaler and prefix-sharded caches reduce origin load and the migration to SQL successfully cut backup and query bottlenecks.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 1:26Have I Been Pwned history
3. 9:55Infrastructure and traffic scale
4. 16:04Cloudflare caching limits issue
5. 22:36Azure Tables to SQL Hyperscale
6. 27:05SQL Hyperscale parallel queries
7. 32:43SQL Hyperscale cost fixes
8. 40:26Cloudflare Workers edge processing
9. 44:44UI speed vs user experience
10. 46:29System observability monitoring
11. 51:00Key takeaways

" related talks

• 37mHave I Been Pwned: Serving billions of requests and terabytes of data without going broke!Stefán Jökull Sigurðarson · NDC Oslo 2023
• 50mHave I Been Pwned?Troy Hunt · GOTO Aarhus 2023
• 56mWhat We’ve Learned From Billions of Security ReportsScott Helme · NDC Sydney 2018
• 51mWhat We’ve Learned From Billions of Security ReportsScott Helme · NDC Oslo 2018
• 31mStandard Classification: Live Q&A SessionTroy Hunt · OWASP 20th Anniversary 2021
• 01h 00mBuilding Safe Systems at Scale - Lessons from Six Months at YahooAlex Stamos · Black Hat USA 2014
• 01h 00mKeynote: How I Met Your DataTroy Hunt · NDC Security 2024
• 01h 00mHyper Speed: When Big Data BloomsScott Helme · NDC Security 2022