Transcend: Detecting Concept Drift in Malware Classification Models
summary
Builds Transcend, a deployable conformal evaluator that computes nonconformity scores (distance-from-centroid or SVM margin) and per-class p-value thresholds to flag unreliable malware predictions before accuracy drops. Using linear SVM on Drebin and Marvin Android datasets it marks >2,000 previously misclassified malicious APKs as unreliable under constraints (target F1 0.99, 76% declareable), then retrains on those labeled samples to recover high precision/recall. P-value gating yields ~90% true positives on “reliable” items versus 66% when using classifier probabilities.
AI-written summary · about
chapters (4)
- 0:00Introduction to concept drift
- 4:26Conformal evaluator framework
- 11:16Experimental results
- 22:19Q&A and conclusion
related talks
30mData Driven Security
27mDetecting Security Posture Drift in AI-Driven Software...
40mDemystifying File Similarity for Malware Detection
27mWhen Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks
27mA Machine Learning approach for detecting a Malware
31mUncertainty quantification: How much can you trust your machine learning model?
31mAn AI Approach to Malware Similarity Analysis: Mapping the Malware Genome With a Deep Neural Network
26mWhen Models Write Malware: Defending Against On-demand Exploits