The Art of Web Scraping. Trees, Algorithms and JS Magic
summary
A presenter built a small DSL parser and an NPM package called харвестер that turns template strings into tree structures, then fuzzily matches those trees against the page DOM using a score heuristic to extract fields as JSON. The tool runs inside Playwright or Puppeteer for live scraping (with stealth browsers to avoid bot detection), and was demonstrated extracting images, titles and prices from Rozetka and Amazon while tolerating many DOM changes; claimed optimizations yield large speedups (up to 100–130,000×) with examples like ~15 s for a full product extract and ~200 ms per complex Amazon block.
AI-written summary · about
chapters (11)
- 0:00Web scraping with JS
- 3:55Traditional data collection issues
- 6:26DSL for fuzzy data search
- 10:57Tree comparison in scraping
- 17:38Search algorithms and scoring
- 20:48Library performance optimization
- 27:32Comparison with AI solutions
- 29:20Live scraping demo
- 35:29Playwright & Puppeteer integration
- 39:46Library API features
- 41:53Q&A
related talks
01h 01mLevelling Up Your Web Scraping Game
26mWeb Scraping with Puppeteer and NodeJS
38mScraping LEGO for Fun: A Hacky Dive into Dynamic Data Extraction
33mBecoming a Spider
46mMastering Web Scraping: Unleash Your Data Extraction Wizardry!
01h 01mWebcrawler
24mimprov BOF How to Scrape Data from any Website
20mWebMCP: Stop scraping the Web, start calling it