Using Promptable Image Analysis for Clinical Diagnostics
summary
Kaiko.ai builds a multimodal clinical assistant called Kao Insight that fuses ViT image encoders with vLLM-style language models, deploys training on Ray Train and serving on Ray Serve inside AKS, and validates outputs with on-site medical ambassadors. Models live on hospital tenants with per-cluster audit trail and managed identities, most models scale to zero to save budget but cold starts can take minutes, and the training fleet recently grows to roughly 64 H100 GPUs for foundation-model work.
AI-written summary · about
chapters (27)
- 0:00Introduction to Kao and team
- 1:06Medical data and clinical challenges
- 2:30Kao Insight multimodal assistant
- 5:45Foundation models and Ray setup
- 6:07Ray in model development cycle
- 7:11Deploying models in production
- 9:57Clinic integration difficulties
- 12:26Training setup design principles
- 13:10Audit trail and data access
- 14:37Kray tool for cluster access
- 15:19Runtime environment resolver
- 16:21Development cluster setup
- 16:42Serving models in hospital tenants
- 17:47GPU resource management
- 18:10Smooth transition to production
- 18:53Ray Serve infrastructure
- 20:00Ray Serve application workflow
- 21:01Onboarding new models
- 22:05Clinical model validation
- 22:46Validation approaches
- 23:51Challenges solved with Ray
- 24:33Accessibility for engineers
- 25:15Authentication and Ray clusters
- 25:56Logging with Ray and Fluent Bit
- 26:38Open challenges with Ray
- 28:42Conclusion and future work
- 29:25Q&A session
related talks
03mAI solutions for healthcare
30mFrom Code to Clinic — Building an Internal Developer Platform for Clinical AI
47mAI for making decisions in clinical medicine
15mAI-Ready Microservice Architecture
31mFrom Model Serving To Distributed Inference: How Llm-d Evolves AI Platforms on…
26mFrom Pixel to Patient: Making Medical AI Work in the Real World
29mServing the Future: KServe’s Next Chapter Hosting LLMs & GenAI Models (with Fun Drawings!)
28mAI in healthcare - Deep learning for precision psychiatry