From Resque to SolidQueue - Rethinking our background jobs for...
summary
Migration proves practical: code changes are a one-line ActiveJob override that routes new jobs to SolidQueue while older jobs stay in Resque, enabling a parallel rollout. Production uses a separate MySQL queue database (dev/staging share primary), SolidQueue requires increasing text column size for huge arg blobs, bumping worker memory and DB connections fixes unexpected crashes, and Prometheus metrics plus SLO-named queues like within_1_minute let autoscaling meet targets while handling 3 million jobs a day.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 4:14SolidQueue replacement choice
- 7:21Transactional integrity setup
- 10:12Latency-based queue naming
- 13:26Worker config and queue monitoring
- 15:10Resque to SolidQueue migration
- 18:21Infrastructure under-provisioning fixes
- 21:10Queue SLOs for performance
- 22:31Delayed jobs and DB load
- 24:39Migration results
- 26:47Q&A
related talks
20mSolid Queue - Default Active Job Adaptor in Rails 8
28mFair by design: orchestrating background jobs in Ruby
39mAsynchronous Workers to the Resque
35mProcesses and Threads - Resque vs. Sidekiq
35mSidekiq at Gusto: The Parts Where People’s Jaw Drops
41mInside ActiveJob
27mScaling a Rails Monolith w/Event Driven Microservices
20mSafely Decomposing a Highly Available Rails App