Measuring End-to-End Security Engineering
summary
Twilio formalizes "End to End Security Engineering" by defining a perfect secure system and turning threat models into measurable artifacts: % of products threat modeled, % attacks monitored, % attacks with controls, % controls covered by tests, plus # successful and prevented attacks. A biweekly bug postmortem traces when bugs are born, assigns theoretical bounties, measures bug survival and process failures, and feeds fixes into SDL. A prototype tool called Blackbird maps microservices, attacks, controls and validation confidence so teams pick controls that reduce residual bounty most cost‑effectively.
AI-written summary · about
chapters (11)
- 0:00Start
- 1:08How Do You Know Your Security Program Works?
- 8:04First Metric
- 10:13Bug Postmortems
- 13:03Estimated Bounty Priority
- 14:35Are we producing more bug-free features?
- 14:50Are we finding bugs faster?
- 15:12What parts of our SDL need to be improved?
- 16:59Wait, won't those stats change in six months if additional bugs are found for those features?
- 29:35What's Next?
- 31:51Experiments • How long were the developers with the company when they wrote the bug?
related talks
48mRedefining Threat Modeling: Security Team Goes on Vacation
24mBeing secure by design: Engineer-led security
52mContinuous Security
40mMaking Security an integrated part of the SW Dev Lifecycle
51mThe End of DevSecOps?
57mBuilding a Modern Security Engineering Organization
53mA DevSecOps Tale of Business, Engineering, and People
41mTest Smarter Security Essentials for Quality Engineers