Background Processing with
summary
A presentation about Square Enix’s open-source TaskBunny system for background jobs, showing a RabbitMQ-based queue, retry and backoff logic, and an OTP supervisor pattern that automatically reconnects and recovers connections. It presents live numbers and tradeoffs—throughput, retry behavior under load, how many concurrent workers run before latency spikes—and demos queuing, backoff, and RabbitMQ integration with concrete failure modes handled.
AI-written summary · about
chapters (8)
- 0:00Background processing intro
- 7:17Message brokers
- 13:51Queueing systems
- 20:00Handling user requests
- 27:59Why background processing
- 33:52Optimizing task performance
- 40:07Scaling solutions
- 46:45Q&A
related talks
01h 05mScaling Applications with RabbitMQ
41mDecouple all the things: Asynchronous messaging keeps it simple
27mRuby Messaging Patterns
02h 59mPractical RabbitMq workshop
39mAsynchronous Processing for Fun and Profity
02h 45mThe world of High Performance Distributed & Async Task Queue(s) with Celery
32mRabbitMQ in Real Life - Tips and Tricks
39mReal World RabbitMQ: Exploring Practical Applications