Accelerated Big Data Systems - Experiences with OpenCAPI
summary
Experiments run on a POWER9 OpenCAPI system show an end-to-end FPGA toolchain: a Parquet-to-Arrow decoder achieves about 12 GB/s using the 512-bit OpenCAPI link (targeting ~20 GB/s with 1024-bit and timing fixes), a Snappy decompressor offloads CPU work, and an HBM buffering layer feeds parallel kernels. Interfaces are generated by Fletcher and on-chip data layout uses the Tidy wire specification. A Wikipedia full-text demo combines decompression plus three to five parallel kernels and records about a 5x speedup versus a dual-socket CPU. Most code is open source on GitHub.
AI-written summary · about
chapters (13)
- 0:00Start
- 0:34Outline
- 1:15Accelerated Big Data Systems
- 2:06Software
- 13:00Fletcher: Interface generator
- 14:49Tydi: Typed Dataflow Interfaces
- 19:00Snappy decompressor
- 20:32WIP: HBM buffer
- 21:18FPGA design toolbox
- 22:13Tools
- 23:43Hardware platform
- 24:09Results
- 28:03Open POWER Summit
related talks
43mA course on Accelerating Big Data Analytics Application with...
07mData analytics accelerated with FPGA's
01h 02mApache Arrow: High-Performance Columnar Data Framework
45mAccelerating OpenSearch With Streaming: Apache Arrow, Flight, Data...
30mIntroducing Spark Cyclone Accelerating Spark with...
30mProject Caerus: Compute Storage Coordination
01h 28mTaming the data demon using HPCC systems
22mData transport in a radio telescope: Remote Direct Memory Access over Ethernet from FPGA to GPU