It's in the Air Tonight. Sensor Data in RAG
summary
Builds a runnable Python pipeline that pulls sensor REST feeds and traffic and weather data from openAQ, enriches and vectorizes readings, and loads them into Milvus for semantic + filtered search. Uses embeddings from Hugging Face, retrieval with LangChain, and local model hosting via Ollama. Shows index choice (IVF_FLAT) and multi-vector hybrid queries combining image and text, pushes formatted air quality reports to Slack, and demonstrates it actually returns sensible, debuggable answers on a modest laptop.
AI-written summary · about
chapters (11)
- 0:00Welcome
- 1:04Sensor data and RAG overview
- 6:13Architecture and data flow
- 17:56Sensor data challenges
- 25:48Data ingestion and storage
- 35:27Milvus setup and schema
- 46:07LangChain introduction
- 59:41Live demo and real-time
- 1:15:39Querying and reports
- 1:20:40Optimizing search accuracy
- 1:28:46Closing and Q&A
related talks
47mReal-Time AI with Open Source
01h 35mTutorial: Build AI-Supercharged RAG Apps with a Vector Database
33mVector Databases in the Age of AI: Embracing RAG for Enhanced Search Capabilities
46mAdvanced RAG with Azure AI Search - top questions answered
49mOrchestrating data and ML workflows with Apache Airflow
41mGenAI, vector/semantic/hybrid search, RRF, NLP, (...) and other buzzwords
16mOpenRAG: An open-source stack for RAG
41mAdding Generative AI to Real Time Streaming Pipelines