High Performance Computing: The Modern Approach
summary
Argues that memory latency, not CPU clock speed, is the primary performance limiter and recommends measuring performance before adding parallelism. Describes concrete compiler optimizations (loop unrolling, function inlining) and SIMD vectorization using CPU registers, plus GPU programming with OpenCL/CUDA while warning GPUs help only when computation far outweighs data-transfer overhead. Gives practical examples of cache-line layout, data layout decisions, and thread contention that show real tradeoffs and cases where parallelizing can slow execution.
AI-written summary · about
chapters (13)
- 0:00HPC importance
- 3:09HPC challenges
- 8:45Processor speed vs memory
- 13:03Cache optimization
- 18:15Compiler optimizations
- 23:34Vectorization and SIMD
- 25:40GPU in parallel computing
- 27:46GPU limits and algorithms
- 29:11Performance measurement
- 31:46C++ in HPC
- 32:51Physical processor limits
- 34:18Vertical integration
- 34:59Parallelization limits
related talks
01h 05mPerformance Optimization in Software Development - Being Friendly to Your Hardware
21mHPC Green Computing: Relationships Between Throughput and Latency
01h 04mDesign for Performance
01h 37mAn Overview of Program Optimization Techniques
01h 06mPerformance engineering: being friendly to your hardware
50mDesigning for Performance
51mDesigning for Performance
36mWhy Are GPUs (Not) Fast - A Trip Through the Driver Stack