2014.10.29 - Security@Scale - Louis Brandy, Facebook
summary
Describes a latency‑sensitive rule engine built in Haxl that executes every synchronous Facebook and Instagram check, batching independent data fetches across Thrift, mcash and graph APIs to avoid N+1 churn. Implements resumable computations with continuations and applicative composition so dozens of rules run in parallel rather than sequentially; hundreds of thousands of lines of rules run live. Claims logins shrink from ~40 seconds to ~100 ms and the system is open source. batching fetches resumable computations thrift + mcache
AI-written summary · about
chapters (34)
- 0:00Intro
- 1:15Background
- 2:31Where would I start
- 2:58Agenda
- 3:54Manual investigation
- 5:02Teaching computers to find evil
- 9:11What is a rule
- 10:01Supplementary data fetching
- 11:22Scalabilitysensitive rule engine
- 13:11Data fetching becomes pure
- 13:36Fewer functions are awesome
- 13:50Mathematical consequences
- 14:52Example
- 19:37Challenge
- 20:38Syntax
- 21:39Hackcyl
- 22:17Why Haskell
- 22:44Why Scala
- 23:06Intersect problem
- 24:17Intersect function
- 25:31Highschool function
- 26:27Computation
- 27:57Composing Futures
- 30:42Thrift
- 31:30Other Thrift Services
- 31:47Rule Engine
- 32:50Open Source
- 33:07Rules
- 35:07Machine Learning
- 37:32Instagram
- 39:44Timing out
- 41:15Historical data
- 42:07Catching throwing errors
- 43:49Batch API
related talks
36mSpam-fighting Infrastructure
12mWelcome to Spam Fighting @Scale
55mFacebook's Approach To Common Web Vulnerabilities
16mPreventing Hacks with Security Rules Written in Solidity
26mRise of the Machines Security Automation at Twitter
31mPlatform Integrity: Lessons Learned in Protecting the Facebook Platform
38mBuilding Secure Systems in Haskell
29mScaling Grid Security With Open Source