Building an Agentic Platform
summary
Box moves from single-shot LLM extraction to an agentic-first design that orchestrates OCR, multi-model voting (OpenAI, Anthropic, Gemini, Llama), page-image checks and directed graph routines to handle 100+ field, 300‑page documents. A judge model plus eval and challenge sets retries low‑confidence outputs, trading latency for enterprise accuracy across an exabyte-scale corpus used by 115,000 customers (two-thirds of the Fortune 500). The platform exposes an agent API and uses LM as judge rather than widespread fine-tuning.
AI-written summary · about
chapters (13)
- 0:00Box's Content Platform and Enterprise Focus
- 1:50Initial AI Deployment in 2023
- 2:54The Challenge of Unstructured Data in Enterprises
- 3:56Limitations of Pre-Generative AI Data Extraction
- 4:54First Version: LLM-Based Extraction
- 7:05Challenges with the Pure LLM Approach
- 8:58Despair and the Need for a New Architecture
- 9:30Introducing Agentic Architecture
- 10:04AI Agent Reasoning Framework
- 10:45Agentic Routine for Data Extraction
- 12:28Advantages of Agentic Architecture
- 14:05Key Lesson Learned: Build Agentic Architecture Early
- 18:37Approach to Fine-tuning and Model Support
related talks
21mDemocratizing AI with Open Source Multi-Agent Systems
33mDeep research at scale
34mKeynote: Building for AI at Scale: Infra, Integrity, & Innovation
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
30mAI Application Architecture: Building for Scale and Efficiency
33mAgentic AI – What's Next?
29mGenerative-AI: Usecase-Specific Evaluation of LLM-powered Applications
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI