Future Open Source LLM Kill Chains
summary
Claim: an advanced, patient adversary can weaponize open-source LLMs and their supply chain by becoming maintainers, publishing tainted models, or adding malicious libraries; a documented case registers a fake CLI and gets 30,000 downloads in one month. Proof-of-concept work fine-tunes a code model to embed an “Easter egg” that exfiltrates credentials when loaded. Practical attack vectors: pickle-based model serialization (remote code execution), unpinned transit deps, and model blobs too large for scanners. Concrete mitigations: use poetry lock files, prefer safe tensors, clone models over SSH/Git LFS, run red teaming and automated tests like GAK, and filter agent inputs/outputs.
AI-written summary · about
chapters (9)
- 0:00Open Source LLM threats
- 6:28Cyber kill chains in AI
- 10:02Adversaries hijack projects
- 16:12Malicious code in downloads
- 19:23Poisoning models with formats
- 24:50Data poisoning risks
- 26:13LLM contamination experiments
- 28:00Mitigations and controls
- 31:09Call to action
related talks
44mI Abuse Your LLM Live with 10 Security Vulnerabilities
49mDoors of (AI)pportunity: The Front and Backdoors of LLMs
50mMalice In Chains: Supply Chain Attacks Using Machine Learning Models
56mHidden Risks in Open-Source Code and AI Models
28mSecuring LLMs: Insights into OWASP top 10
49mBreaching LLM-Powered Applications: Overcoming Security and Privacy Challenges
36mCompromising LLMs: The Advent of AI Malware
39mLLMs everywhere: The future is now and the past keeps repeating itself