Hands-on Multimodal AI Development with Pixeltable
summary
Pixeltable claims a declarative, incremental data layer for multimodal ML that version controls tables, materializes video into frames, and runs model inferences as declarative incremental computed columns. Demos use a local Postgres instance installed by pip, built-in UDFs and custom Python UDFs, YOLOX for detection, CLIP for image/text embeddings, Whisper for transcription, and a Gradio/Hugging Face app. Embedding indices are maintained automatically as collocated embedding indices, with code and notebooks on GitHub.
AI-written summary · about
chapters (13)
- 0:00PixelTable introduction
- 1:06Multimodal AI challenges
- 2:32PixelTable principles
- 4:16Technical architecture and supported data types
- 6:03Creating tables in PixelTable
- 8:10Vector search in PixelTable
- 10:16Incremental updates in PixelTable
- 11:23Extensibility with custom UDFs and workflows
- 13:09Version control in PixelTable
- 14:56Future roadmap and open-source focus
- 15:43Gradio app with PixelTable
- 18:12Social media from video
- 20:08Q&A
related talks
37mBuilding Multimodal AI Agents From Scratch
01h 34mHands-on AI workshop: Multimodal agents
20mShiny x AI
46mBypassing the Multimodal Tax: Hybrid RAG, SQL RRF & UI Telemetry
01h 30mBuilding Your First Multimodal Gen AI App 🚀
40mRolling out Enterprise AI: Tools, Insights, & Team Empowerment
56mBeyond Chat: Bringing Models to the Canvas
55mEveryday AI: Practical Applications Beyond the Chatbot