Six Little Lines of Fail
summary
Six lines of code call a database, Stripe, SendGrid and RabbitMQ inside one DB transaction and that hides real failures: customers get charged but no order exists, emails never arrive, downstream systems never see messages. Practical fixes shown: Stripe idempotency or authorize then capture to avoid double charges, SendGrid treated as at least once or ignored, RabbitMQ solved with an outbox pattern, and complex coordination moved to a background process manager; the short-term choice is refund on failure.
AI-written summary · about
chapters (25)
- 0:00Intro
- 1:246 little, innocuous lines of code
- 3:56Failure is ALWAYS an option
- 10:41is probably not good for business
- 12:12We can't put the world into a transaction
- 12:23Assume failures
- 12:464 options for dealing with failures between two systems
- 18:05The Master Plan
- 18:52First up: Stripe
- 19:26Probably don't want to ignore a failed payment
- 20:12Retry
- 24:01Undo
- 30:51Not unless we want duplicate emails
- 31:45We can't un-send emails
- 33:02Finally: RabbitMQ
- 35:23Can't un-send a message
- 36:11Coordinate
- 37:15Taking a step back
- 37:33Can we avoid exceptions by breaking the process coupling?
- 42:49Way too much going on in a single button click
- 45:11Starting the workflow
- 45:55Routing slip
- 46:50Saga pattern
- 47:51Process manager pattern
- 48:40Orchestration or choreography?
related talks
01h 02mSix Little Lines of Fail
01h 02mFive things every developer should know about building mission-critical systems
34mNightmare Before Best Practices
31mThe Anatomy of a Cascading Failure
42mHow to stay alive even when others go down:...
42mPlanning To Fail
48mFault Tolerance on the Cheap Making Systems That Probably Won't Fall Over
52mPerformance & Stability Anti patterns