Building a Multimodal Search Engine
summary
A practical Python pipeline that builds multimodal embeddings for text and images, storing them in PostgreSQL vector columns and performing similarity search for retrieval-augmented generation. The pipeline uses Amazon Bedrock with Titan for embeddings, converts images into base64 image embeddings, chunks text and video frames, links transcribed audio by timestamps, and runs a demo (code on GitHub); relevance checking is done manually in this project and costs are driven by token usage.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:36AI context
- 7:28Foundation models
- 11:03Multimodal models
- 14:38Implementation challenges
- 16:23Demo and conclusions
- 17:06Q&A: Democratization
- 20:17Q&A: Open-source costs
- 21:20Q&A: RAG verification
related talks
26mAdvancing Multi-Modal Search Capabilities in Search Pipeline
27mMachine Learning for Advanced AI Search
57mUsing Vector Databases for Multimodal Embeddings and Search
29mA Pythonic semantic search
01h 08mGoing multi-modal
25mSemantic search with Django, PostgreSQL, & pgvector
01h 28mBuilding a Semantic Search Engine
41mSemantic Multimodal Multilingual Similarity Engine