Heisenbug
summary
A staging app returns a 500 error for an unknown path even though a catch all Cowboy handler should serve a 404 the logs are empty and the stack trace shows an undef calling inspect with two args only in production after protocol consolidation further investigation reveals a mounted compiled slug with OverlayFS and a cleanup script that hides beam files in /tmp causing modules to be missing at runtime the failure is non deterministic based on whether the VM has already loaded the module the fix is to stop cleaning or move the overlayed temp directory
AI-written summary · about
chapters (7)
- 0:00Start
- 0:10IT STARTED with a 500
- 0:32The logfile
- 1:21the offending code (?)
- 2:14Debugging results
- 3:33The answer (tidr. it was our fault)
- 5:02Solution
