IP Protection and Privacy in LLM: Leveraging Fully Homomorphic Encryption
summary
Claims a practical hybrid method using Fully Homomorphic Encryption and Zama’s Concrete tools to keep user queries private while keeping model weights server-side as IP; uses TFHE primitives and table lookups, with programmable bootstrapping under a second. Demo runs LLaMA-3 7B with tokenized inputs compressed to ~90 KB and server outputs about 40 MB, doing FHE inference in ~0.8 s but yielding ~1 token every 3–4 s because of transport and heavy 4096 ciphertexts per token attention multiplications; code and blog are open source.
AI-written summary · about
chapters (21)
- 0:00Privacy in LLMs
- 1:04What Zama does with fully homomorphic encryption
- 2:30LLM privacy risks
- 5:44Limitations of private LLMs
- 7:15FHE for privacy and IP
- 8:01What is FHE
- 10:33FHE for ML and LLMs
- 11:59Basic operations in LLMs and FHE feasibility
- 13:06Tokenization in encrypted LLMs
- 14:36Feed-forward layers in FHE
- 16:31Attention bottleneck
- 19:49Hybrid FHE LLMs
- 20:54Where intellectual property resides in LLMs
- 23:46Client-server LLM split
- 25:35Encrypted LLM demo
- 28:33Demo results
- 30:44Key takeaways
- 32:12Adopting privacy-preserving ML
- 33:18Try FHE LLMs
- 34:46Encrypted LLM use cases
- 35:29Q&A
related talks
52mMachine Learning over Encrypted Data with Fully Homomorphic Encryption
40mTrust No One
20mDeep Dive into Privacy Enhancing Technologies
37mOpen-source Machine Learning on Encrypted Data
33mThe Promise of Trustworthy AI
08mLLMs Without Leaks: Keep Your Code, Data & IP
01h 07mLattices and Encrypted Computation
06mLightning Talk: Multi-Party Learning Using Homomorphic Encryption with Kubeflow...