Unbounded Concurrency in Go
summary
Progressively builds a TCP port scanner in golang that starts with net.Dial to test single ports, scales to a loop over ~300 ports, and then goes concurrent with goroutines where unbounded launches hit “too many open files” errors. Fixes include sync.WaitGroup for waiting, a CPU‑sized worker pool, a semaphore from x/sync/semaphore (example uses max weight 100000 acquire 100 which gates ~1,000 goroutines), per‑port contexts with timeouts, and a pipeline with fan‑out/fan‑in and a done channel; the final tools work and avoid resource exhaustion.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:37Overview
- 2:42Code
- 8:34Sync Package
- 11:51Worker Pool
- 15:56semaphore
- 21:15pipeline
- 27:36merge
- 35:56Wrap up
- 36:26Livestream
- 37:42Concurrency Recommendations
- 40:10Concurrency Tips
- 43:19Channels
- 44:40Tools
related talks
58mUnbound Concurrency in Go
31mConcurrent Server Architectures
24mMastering Concurrency in Go: From Patterns to Production
29mGoing Infinite: Handling 1 Million Websocket Connections in Go
41mDetecting and Fixing Unbound Concurrency Problems
31mGo: Code that Grows with Grace
21mReaching the ceiling of single-instance Go
35mGo’s Trace Tooling and Concurrency