Playing tasks with Django-Celery
summary
Uses Celery with a RabbitMQ broker and MongoDB result backend, runs Django-Celery (jungle celery) for web scheduling and admin, and adds a small web view to trigger registered tasks. Implements per-task max_tasks_per_child to avoid memory leaks, a Mongo-backed unique-task lock on arguments to prevent concurrent work on the same album, and custom task base class methods (override __call__ and after_return) for temp folders and cleanup. Centralized logging goes to syslog.
AI-written summary · about
chapters (7)
- 0:00Start
- 9:22Celery worker
- 13:52Defining a simple task
- 15:43Retrying a task if something fails
- 17:46Task set example
- 22:07The Jamendo Task class
- 30:27Jamendo needs UNIQUE tasks
related talks
40mHow to Schedule Tasks with Celery and Django
32mFrom crontab to celery with no regrets
49mIntroduction to Celery
28mBoosting Python Web Apps performance with Celery and Concurrency
19mOptimizing Data Retrieval with Python Celery
36mAn Introduction to Distributed Task Queue
24mUsing Python, Django, and ruthlessness to protect people from social media harassment.
29mSwiss Army Django: Small Footprint ETL