Building Workflows With Celery
summary
Celery runs distributed task workflows by sending serializable signatures and primitives (chain, group, chord, map) as messages to workers that must have the code to execute them. Example: split login work into a device check, parallel notify/save group, then post to an external API and postprocess. Error handling is per-task (retries or link_error callbacks) and groups block until every task finishes—group waits for all tasks; no group-level retry. RabbitMQ is preferred; some workflow bugs persist.
AI-written summary · about
chapters (12)
- 0:00Intro
- 1:07Celery Overview
- 3:39Signatures
- 4:35Serialization
- 5:46Signature
- 7:22Callbacks
- 8:36Chain
- 9:14Group
- 10:52Workflows
- 14:29Handling Errors
- 17:00Handling Different Results
- 18:06Async Results
related talks
02h 45mThe world of High Performance Distributed & Async Task Queue(s) with Celery
25mMixing reliability with Celery for delicious async tasks
45mPlaying tasks with Django-Celery
38mImplementing Celery, Lessons Learned
17mBuild a prod-ready distributed task queue system with celery
38mDistributed Workflows with Flowy
19mDistributed Workflows for Microservices-Style Applications [I]
39mWorkflows, a new abstraction for distributed systems