Complex Data Ingestion with Open Source AI
summary
Claims an open-source toolkit named Docling reliably converts messy PDFs, scans and slides into structured JSON/markdown/HTML, preserving headers, tables, images, captions and page coordinates; code-along labs show conversion, hybrid chunking that balances token limits with document hierarchy, and a multimodal RAG pipeline with visual grounding. Tools used include OCR, VLMs (Granite Docking 256M–8B), LangChain, Milvus and Replicate; demo processes a ~20-page doc into ~72 text chunks, 53 image descriptions, retrieves a $13M vs $383M result, and provides runnable Colab notebooks.
AI-written summary · about
chapters (20)
- 0:00Docking and document conversion
- 4:33Workshop setup
- 5:15Docking conversion pipelines
- 7:21Basic document conversion
- 10:54Exporting tables and images
- 12:57Extracting document structure
- 15:27Enriching with visual language models
- 19:22Chunking for AI processing
- 23:59Hierarchical chunking methods
- 27:51Contextualizing chunks
- 30:43Building multimodal RAG pipeline
- 31:26Multimodal RAG lab setup
- 35:16Processing image-heavy docs
- 38:55Visual grounding in RAG
- 43:30Assembling full RAG pipeline
- 46:20Real-world use cases
- 48:27Handling handwritten/scanned docs
- 52:19Addressing hallucinations
- 56:57Parsing with screenshots
- 59:49Q&A
related talks
01h 25mLearn to Unlock Document Intelligence with Open-Source AI
01h 19mLearn to Unlock Document Intelligence with Open-Source AI
26mAdvanced Document Parsing for AI Workflows
01h 07mMeet Docling: The “Pandas” for Document AI
22mStructuring the Unstructured : Empowering the AI Workflows
48mIntegrating Docling With OpenSearch for Advanced RAG and Agentic Applications
31mDocling: Get Your Documents Ready for Gen AI
01h 30mUnlock Information from Tables, Images and Complex Docs