Parallel Sequences in Multicore OCaml
summary
An OCaml 5.0 multicore library that exposes list-like tabulate/map/reduce/scan/flatten/filter uses an opaque array underneath, domainslib for scheduling, and a small C stub to allocate uninitialized blocks; it achieves good speedup on 2000×2000 block matrix multiply with 100×100 blocks and on a web index built from 1,000,000 pages (~800 MB), and an in-place scan removes array.copy to scale on an array of 1,000,000,000 integers; main costs are domainslib’s scheduler choices and sequential array initialization plus GC marking quirks.
AI-written summary · about
chapters (17)
- 0:00Parallel sequences intro
- 1:27Core parallel ops
- 2:30Shared memory in Multicore OCaml
- 3:33Tabulate with domains
- 6:21Array init challenges
- 7:03Reduce and scan algorithms
- 8:47Flattening nested seqs
- 9:29Parallel filtering
- 10:31Block matrix perf
- 11:35Web indexing
- 12:38Domainslib limitations
- 13:40OCaml runtime init
- 14:26Array creation overhead
- 15:50In-place operations for better performance
- 16:54Array of arrays workaround
- 17:36Garbage collection considerations
- 17:57Conclusion
related talks
40mWhen magic meets multicore - OCaml and its elegant era of parallelism
19mConcurrent Data Structures Made Easy
18mMulticoretests - Parallel Testing Libraries for OCaml 5.0
25mPriodomainslib: Prioritized Fine-grained Parallelism for Multicore OCaml
34mFrom OCaml 4 to 5 and from Parmap to Effects: A legacy code transition story
37mScalable Parallel Algorithms
41mParallel Programming, Fork Join, and Reducers
15mCosmo: A Concurrent Separation Logic for Multicore Ocaml