Elevating enterprise data through open source LLMs
summary
Shows a working architecture that keeps enterprise data and models inside Databricks while exposing a secure Posit Connect Python Shiny app for users, using RAG with vector DBs, Databricks model serving and inference tables, langchain and MLflow for chains and logging, and SSO-based access control so answers respect user permissions. Demo confirms per-user access differences, automatic request/response logging for auditing and model-improvement, and recommends permissive open models (e.g., Llama 3.1) for end-to-end control.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:45Competitive Advantage
- 1:56AI
- 2:18Rag
- 4:53Security
- 6:00Components
- 6:50tooling
- 7:28demo
- 8:46colleague view
- 9:42inference table
- 11:02architecture review
- 11:51SSO
- 12:48Summary
- 13:59Questions
- 15:54Examples
related talks
52mElevate your data and AI governance with Databricks Data Intelligence Platform
19mFrom SDKs to Agents: Building with R and Python on Databricks
16mKeynote: DBRX: Training Modern LLMs on Databricks
29mML in Production on Databricks
21mPosit and the modern data stack: Using Posit, Snowflake, and Databricks
39mSelf-Service Analytics using LLMs
34mOverview of the Databricks Platform
49mGenAI for Business on the Modern Data Stack