Building Scalable & Resilient Background Jobs for FinTech
summary
Ruby on Rails pipelines that schedule region-specific payouts via a cron scheduler create database-backed job rows, queue them into the culus gem worker system, and enforce dependencies with a deliberately created noop job for time plus predecessor constraints. Idempotency uses safe-point idempotency—unique labels persisted to skip already-completed steps—while retries include automatic query-retries with doubled timeouts. Observability uses a DataDog cutoff metric, Slack alerts, per-job SLAs, and TTL/archival of old executions so on-call can intervene only when thresholds break.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:30Background job pipelines in FinTech
- 7:26Pipelines with Ruby and Qless
- 12:48Scheduling and executing jobs
- 14:15Resiliency in job systems
- 17:49Observability and monitoring
- 22:29Health checks and alerts
- 23:37Q&A
related talks
11mRunning Jobs at Scale
27mPrioritization justice: lessons from making background jobs fair at scale
32mZero Downtime Deploys Made Easy
39mHacking Sidekiq for Fun and Profit
23mLook Ma, No Background Jobs: A Peek into the Async Future
44mMo' Jobs Mo' Problems: Lessons Learned Scaling to Millions of Jobs an Hour
27mPayment workflow with Ruby and Clickhouse
28mAcidic Jobs: A Layman's Guide to Job Bliss