Internet Archive: More than the Wayback Machine
summary
Nonprofit library with about $12M/year budget, roughly 50 petabytes of spinning disk (duplicated) and 32 scanning centers that scan ~1,000 books/day, the archive serves ~2M daily visitors and the Wayback gets ~600k/day. Available tools include a Wayback machine API (URL checks, timestamped nearest snapshots), an 80 TB crawl dataset, an Open Library metadata API and an S3‑compatible IA S3 API for uploading, programmatic metadata edits, and full‑text and captioned TV archives. Protocols: robots.txt respect and hashed IPs for privacy.
AI-written summary · about
chapters (10)
- 0:00Start
- 1:15Alexis Rossi VM Brasseur
- 2:13Some Handy Links
- 21:20Wayback Machine API
- 24:03Broken Link Checker
- 24:50404 Handler
- 25:23Open Library API
- 27:29Evergreen
- 28:17OpenBook WordPress Plugin
- 31:43IA Metadata API
related talks
31mThe Web We Want, The Web We Deserve
25mScanning is the new spinning - Digitization & Open Knowledge initiatives around you
52mLibrary of the Future
11mHow to stop worrying & collect early banner ads!
53mIntroduction to Software Heritage, the Alexandria library for code
15mWhat will remain of my website in 50 years?
40mSoftware Heritage
27mInternet Jones and the Raiders of the Lost Trackers...