Elixir With
summary
Claim: achieving five nines availability comes from rigorous monitoring plus eliminating single points of failure, not vague best-practices. Concrete tools are metrics, logs and alarms; concrete numbers and failures include a message-queue spike to over 1,554,000 messages with a four-hour recovery, a backlog of half a million reauthentications, crash dumps lost to restart scripts, and an ETS table limit 2053 capping concurrent logins. Examples show MongoDB driver mismatch, 200 VMs, and practical fixes like aggregated logs, threshold alarms and automated remediation.
AI-written summary · about
chapters (14)
- 0:00Intro
- 4:42ATT Long Lines
- 6:48Redundant Hardware
- 10:22Maintenance
- 12:22Monitoring
- 17:05Memory Utilization
- 20:10Message Queue
- 23:12Crash Investigation
- 24:40Log Analysis
- 25:37Crash Reports
- 28:52Audit Trail
- 37:12Alarms
- 41:17Examples
- 47:02Challenges
related talks
56mGetting all the 99.99(9) you always wanted
31mHow We Went From Two Major Outages to 99.98% Reliability in 6 Months
52mPerformance & Stability Anti patterns
30mZero Down Time Operations: How to Get Atleast 3 9's of Uptime
46mMeeting Impossible SLAs: How we made our uptime 99.999%
48mZero Support in Production
45mEngineering resiliency - how I learned to love an outage
44mConcurrency + Distribution = Scalability + Availability(...)