Unbound Concurrency in Go
summary
Builds a TCP port scanner using Go's standard library, starting from a single hardcoded dial to net.Dial and evolving through parallel scans to a worker pool that defaults workers to GOMAXPROCS, buffered channels as a job queue, and sync coordination via sync.WaitGroup. It demonstrates finding port 5432 open, hits the OS "too many open files" limit when naively spawning thousands of goroutines, and shows fixes: bounded workers, a semaphore from golang.org/x/sync/semaphore, and pipeline/fan‑out fan‑in patterns; examples run successfully.
AI-written summary · about
chapters (23)
- 0:00Go concurrency and port scanning
- 3:10Port scanner and Go's role
- 7:46Sequential port scanning
- 9:12Why sequential scanning is inefficient
- 10:16Goroutines without sync
- 12:03Why goroutines need sync
- 13:05sync.WaitGroup introduction
- 18:25First concurrent scanner with WaitGroup
- 19:50Unbounded concurrency risks
- 21:13Worker pool to limit concurrency
- 24:06Buffered channels for work
- 26:13Directional channels and safety
- 28:42Worker pool walkthrough
- 32:16Testing the worker pool
- 32:36Semaphores for batch processing
- 37:36Pipelines for data processing
- 38:37Port scanning pipeline
- 41:27Fan-out and fan-in
- 45:06Final pipeline implementation
- 46:10Context and cancellation
- 50:03Error filtering in pipelines
- 51:52Testing error handling
- 53:55Key takeaways
related talks
47mUnbounded Concurrency in Go
31mConcurrent Server Architectures
41mDetecting and Fixing Unbound Concurrency Problems
24mMastering Concurrency in Go: From Patterns to Production
34mGo concurrency in the real world
26mConcurrency primitives of Golang - how to use and how not to
27mProduction Ready Concurrency
54mComplex Concurrency Patterns with Go