From Postgres full text search to Retrieval Augmented Generative search
summary
A Rails demo builds an email chatbot called Milly the Mailiphant that progresses from Postgres LIKE to full text search with a GENERATED tsvector column and gin index (query drops from 3.5 ms to 0.062 ms) to semantic search using pgvector embeddings (1536-d vectors, inner product; semantic query 10 ms → <1 ms with hnsw index) and finally RAG via the azure_ai extension to create embeddings and summarize; large emails exceed a 4.5k token limit (7000+ tokens), so the system chunks documents, queries each chunk, rejects "no idea" results, and returns the first grounded answer.
AI-written summary · about
chapters (7)
- 0:00Intro
- 1:05Milly the Mailiphant
- 1:25Basic Search
- 2:40Full Text Search
- 7:09What is Semantic Search
- 11:56Retrieval Augmented Generation (RAG)
- 13:44Demo
related talks
47mExtendable by Design: Building Generative AI Apps with Postgres and Vector Storage and Azure AI
43mFull-Text-Search Explained from A to Z with French Food
31mRetrieval Augmented Generation
33mVector Databases for GenAI applications
24mFrom Queries to Agents: The Next Era of Data Retrieval on PostgreSQL
53mModern PostgreSQL for AI...
44mPostgres and OpenAI - a match made in Azure
29mA Pythonic semantic search