A practical guide to speed up your application with Asyncio
summary
Shows a working Marktplaats scraper that switches from requests + synchronous writes to asyncio with aiohttp and asyncpg, creates ~100 concurrent tasks (one per page), and measures roughly a 3–4× speedup versus sync or 4-thread runs. Covers concrete pitfalls: timeouts, passing items via an async Queue, and a bad surprise where exceptions disappear under run_forever when tasks are held as live objects; the async pipeline ultimately works and writes to Postgres.
AI-written summary · about
chapters (7)
- 0:00Async I/O and Asyncio
- 7:24Asyncio fundamentals
- 16:35Scraping Marktplaats
- 22:51Exceptions in async code
- 27:57Passing data between tasks
- 30:24Timeouts for async tasks
- 33:03Summary
related talks
30mLet's get you started with asynchronous programming
40mOptimize Your Database Operations with Asynchronous Processing
28mBoosting Python Web Apps performance with Celery and Concurrency
27m"Python’s tale of concurrency"
32mEnhancing AsyncIO for Complex Applications
32mAsynchronous Django: Unlocking the Power of Async Programming
30mSpeed Up Your Data Processing
56mLeveraging Async and Await in Modern Python