Enhancing Secret Detection in Cybersecurity...
summary
Fine-tuned a small LLM for secret detection using an LLM-agent labeling pipeline over 100,000 labeled files (cost ≈ $5,000) and LoRA adapters plus model quantization for CPU inference. Baseline regex scores are 56% recall / 32% precision; the 1B‑parameter model hits about 82% recall, 85% precision and meets the <10s per‑file single‑thread CPU target (a 0.5B variant reaches ~71% recall with ~2s per file). Uses Gemini/Qwen for labeling, llama.cpp for inference, and a custom postprocess + fuzzy matching to curb hallucinations.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:46Current secret detection limits
- 8:19Why small language models
- 11:36Generating training data
- 14:47Fine-tuning approach
- 17:59Evaluation and metrics
- 21:15Frameworks and tools
- 22:00Key takeaways and future
related talks
25mFinetuning Large Language Models (LLMs) for Security Log Detections
38mNext-Gen Detection: Harnessing LLMs for Sigma Rule Automation
46mHow Large Language Models Are Reshaping the Cybersecurity Landscape
38mSecurity for AI Agents Using an Ensemble of Fine-tuned... (Lidan Hazout, Bar Kaduri)
21mFrom Data to Precision: Fine-Tuning LLMs for Proprietary Query Languages
01h 49mAI engineering with open access LLMs that lie, curse, and steal
37mSecure by Design: Strategies for LLM Adoption in Cloud-Native Environm...
13mIntegrating Large Language Models into Security Incident Response