An Introduction to Distributed Task Queue
summary
Demonstrates a working SMS-driven distributed task queue built on a single AWS EC2 instance that simulates about 15 servers by running multiple workers, uses Flask, Celery with RabbitMQ as the broker, Nginx and a database, and integrates with the Chikka API for inbound/outbound SMS. Shows concrete queues (keyword, board, outbound), two message workers and a notification worker, a 3‑second async processing delay, and an end‑to‑end demo where messages are queued, processed, and delivered successfully.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:41DSA Example
- 2:51Amazon Example
- 3:57Workflow
- 4:36Traditional way
- 6:24Processing
- 7:36Record
- 8:10Communication
- 10:20Customer Queue
- 11:04Task Queue
- 12:18Application Level
- 13:18Application Demo
- 14:36Technology Issues
- 15:43Create Amazon EC2 Instance
- 19:17Automate
- 24:31Data Flow
- 28:46Simulation
related talks
45mPlaying tasks with Django-Celery
49mIntroduction to Celery
01h 31mIt's the message, stupid: python & amqp
40mHow to Schedule Tasks with Celery and Django
45mPlaying tasks with Django-Celery
24mDistributed Web Scraping in Python
22mDistributed Web Scraping in Python
30mApplication integration patterns for microservices