Scaling Oban Applications
summary
Oban runs as Elixir + Postgres and the talk shows concrete scale fixes: Postgres features (skip locked, jsonb, reindex concurrent, btree dedup) and Oban config changes let a single laptop sustain ~700k jobs/min (they claim 1.4 billion/day and prior tests near 1M/min to 2M/sec in lab). Practical actions include disable triggers for bulk inserts, switch to the PG notifier, use the reindexer plugin, partition/drop old tables, and add a dedicated repo/pool for Oban; the live demo attempts to run the billion‑job workload but fails on stage.
AI-written summary · about
chapters (18)
- 0:00Welcome
- 1:48Why systems scale or don't
- 2:53Elixir and PostgreSQL strengths
- 4:00PostgreSQL as job queue
- 5:28Oban for massive scale
- 6:49Live demo teaser and tuning
- 7:31Tips to minimize DB impact
- 10:27Real problems and solutions
- 11:54Fixing PubSub bottlenecks
- 12:56Disabling triggers for bulk inserts
- 13:38Preventing index bloat
- 15:04Optimizing bulk unique inserts
- 16:49Solving Ecto timeouts under load
- 17:54Managing table bloat
- 19:39Handling connection errors
- 21:47Top tips for scaling Oban
- 22:30Live demo and config walkthrough
- 25:02Closing
related talks
36mBackground Jobs with Elixir: Oban in the Real World
51mScaling Postgres to the next level at OpenAI
25mMigrating from C# to Elixir
50mPostgreSQL Scale-Out at inMobi
51mWhen Postgres Can't, You Can!
48mBuilding Petabyte-Scale PostgreSQL Deployments
25mScaling Postgres to the next level at OpenAI
36mPostgres at any Scale