Lessons learned with asyncio ("Look ma, I wrote a distributed hash table!")
summary
Asynchronous I/O using Python 3.4 asyncio actually builds a working distributed hash table: coroutines, futures and the event loop coordinate networked lookups; streams/transports/protocols handle TCP/UDP. The project, dulus, stores keys hashed with SHA-512, resolves lookups as futures and runs parallel peer queries. Implementation stays compact at 871 lines and achieves 100% unit test coverage, demonstrating a small, testable DHT that really works.
AI-written summary · about
chapters (15)
- 0:00Intro
- 1:22WHAT DOES ASYNCIO DO..?
- 3:13WHAT IS A DISTRIBUTED HASH TABLE?
- 4:12A DISTRIBUTED HASH TABLE (DHT) IS A PEER-TO-PEER KEY / VALUE DATA STORE
- 5:18THE EVENT LOOP
- 9:12Programmers define callbacks to be run when the result of a network call is known.
- 9:33CONFUSED..? DON'T BE, ITS EXACTLY HOW HUMANS THINK ABOUT CONCURRENCY.
- 10:45asyncio avoids potentially confusing and complicated "threaded" concurrency while retaining the benefits of strictly sequential code.
- 11:14How do such tasks pause while waiting for non-blocking network based I/O?
- 11:27COROUTINES (Are FUN!)
- 13:34BUT WHAT ABOUT CALLBACKS? How do I handle the result of a coroutine?
- 16:14A DHT EXAMPLE HASHING, DISTANCE AND LOOKUPS
- 19:54GET() & SET() REQUIRE A LOOKUP. All interactions are asynchronous. Lookups are also parallel (concurrent).
- 20:39WHAT ABOUT NETWORKING?
- 21:55FINAL THOUGHTS... • 100% unit test coverage
related talks
24mA Crash Course in Python 3's asyncio
20mAsync Rust Meets Async Python: How Daft runs user defined Python coroutines from Tokio
18mBuilding Lightning-Fast Apps With asyncio
33mArtisanal Async Adventures
40mAdvanced asyncio: Solving Real-world Production Problems
22mEdges of Python Three Radical Python Hacks for Fun and Profit
01h 00mCoroutine Concurrency in Python 3 with asyncio
27m"Python’s tale of concurrency"