Parallelize your code, one step at a time
summary
A concrete recipe appears: use .NET ConcurrentQueue and a concurrent dictionary to stream large results (avoid buffering millions from Get-Mailbox), declare a status flag or close queue to signal completion, and coordinate per-step throttling via a compiled throttle object (e.g. throttle.getSlot) or PSFramework’s built-in runspace workflows. Demos show begin/process/end per worker, per-runspace variables/functions or modules injected, weighted slots and kill vs graceful stop, with occasional demo flakiness but the constructions work end-to-end.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:50Parallelization problem
- 3:38ForEach-Object -Parallel
- 5:02Parallel jobs across servers
- 6:30Highlander runspace risks
- 10:03Per-step parallelization needs
- 14:09Runspaces deep dive
- 16:40Concurrent queues for data
- 23:10Status flags for tracking
- 26:05Thread-safe counting
- 29:02Throttling runspaces
- 32:16PS Framework workflows
- 33:24Basic runspace workflow
- 36:33Queues auto-completion
- 39:00Workflow resources
- 41:31Adding functions to runspaces
- 44:03Long-running commands with queues
- 46:31Q&A on module commands
- 49:24Per-thread setup blocks
- 51:10Throttling in PS workflows
- 53:48Race conditions and safety
- 56:24Closing remarks
related talks
01h 03mPerformance loop—A practical guide to profiling and benchmarking
55mCode the future, now
55mCode the future, now
01h 26mGetting Ahead By Deleting Code
18mCode That Can't Fail
47mTempting Fate: Patterns & practices for shareable scripts and functions
41mLogging in a DevOps world
01h 16mLevel Up Your PowerShell: Essential Tips, Tricks, and Techniques