Breaking the Layer Barrier: Remodeling Private Transformer Inference with...
summary
BLB restructures Transformer inference by decomposing layers into operators, fusing adjacent linear operators under CKKS to avoid frequent HE↔MPC conversions, and adds the first secure CKKS↔MPC conversion protocol to handle increased ciphertext bit width. It implements a rotation‑efficient fused matmul tailored for multi‑head attention and packs batches to cut HE rotations. Evaluated on BERT‑base / BERT‑large / GPT2‑base, it shows 21× comm reduction vs BOLT and 2× vs Bumblebee, with GPU latencies reduced 13× and 1.8× respectively.
AI-written summary · about
chapters (7)
- 0:00Private transformer inference intro
- 1:00Cryptographic private inference
- 3:52BB framework motivation
- 8:38Limitations of hybrid H/MPC
- 9:26BB framework key idea
- 13:28Experimental results
- 14:32Summary
related talks
57mWIP title: Building Practical Privacy-Preserving Inference Systems
26mRethinking the Transformer: Toward Native Multimodal Architectures
30mDo transformers need all that attention?
15mOptimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
38mFrom Von Neumann to Transformers: An Infra Engineer's Guide to What's Actually Inside an LLM.
41mRepresentations from natural language data: successes and challenges
19mSystem for Scaling Large Language Models
01h 27mChallenges in Making LLMs Safe and Robust