Apify SDK - The scalable web crawling and scraping library for JavaScript
summary
A demonstration of Apify SDK shows a concrete, working crawler: Node code under 100 lines uses Puppeteer or plain HTTP to control Chromium, enqueue requests, and store results locally or on the Apify platform with rotating proxies and autoscaling. The example fetches 10 pages of Hacker News, produces 300 files in the Apify storage, and logs processed pages, proving the library actually crawls, paginates, extracts fields, and writes dataset output.
AI-written summary · about
chapters (9)
- 0:00Intro
- 0:24What is it
- 0:45How to use it
- 1:33What it can do
- 2:55Puppet SDK
- 3:25Puppet crawler
- 5:08Chromium
- 5:27Console
- 6:04Outro
related talks
33mBecoming a Spider
26mWeb Scraping with Puppeteer and NodeJS
45mThe Art of Web Scraping. Trees, Algorithms and JS Magic
57mChrome Extension for Page Scraping & Analysis
20mAutomate everything via NodeJS and Puppeteer
01h 15mPuppeteer can automate that!
30mModern Web Testing and Automation with Puppeteer
21mDirty Little Hacks