Optimizing Data Retrieval with Python Celery
summary
Generic web UI hits browser limits of six concurrent connections so a dashboard with ten visualizations stalls. The team builds a prototype using Python Celery with Redis as broker/backing store (dockerized), Django endpoints that enqueue tasks, return async ids, and a polled status endpoint. Demo runs eight concurrent queries, tasks save results to DB and a Redis cache, and the UI shows incremental tiles quickly — concept works in demo but long queries still need caching and worker scaling.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:36Overview
- 1:16About us
- 3:10Background
- 7:10Solutions
- 8:42Task queues
- 9:53Python Celery
- 15:43Additional Configuration
related talks
28mBoosting Python Web Apps performance with Celery and Concurrency
47mBuilding Scalable Web Apps
40mHow to Schedule Tasks with Celery and Django
49mIntroduction to Celery
45mPlaying tasks with Django-Celery
45mPlaying tasks with Django-Celery
26mBuilding a RESTful real-time analytics system with Pyramid
29mSwiss Army Django: Small Footprint ETL