Concerto for Java & AI - Building Production-Ready LLM Applications
summary
Built a composer assistant application using Spring AI that runs locally on Olama and switches to Mistral AI in the cloud, demonstrating semantic search with embeddings stored in PG vector/Postgres, RAG via advisors, and function calling tools for database queries. Classification is enforced as Java enums with an unknown fallback to mitigate prompt injection and hallucination. Dev observability uses OpenTelemetry and Grafana. Live scoring demo composes and performs music from model-generated plans with minor latency but overall successful results.
AI-written summary · about
chapters (14)
- 0:00Intro
- 2:10The WHY Factor
- 4:35Machine Learning
- 6:38Model Inference via HTTP APIs
- 8:28Building a Chatbot
- 13:37Protecting Against Prompt Injection
- 16:40Semantic Searching
- 24:05Retrieval Augmented Generation
- 28:05AI Model Observability
- 30:45Structured Data Extraction
- 33:07Tools/Function Calling
- 35:19Developing a Composition Plan
- 37:00Writing a Composition Plan
- 44:30Conclusion
related talks
51mIntelligent Applications with Spring AI
50mModular RAG Architectures with Java and Spring AI
46mThe Era of AAP: Ai Augmented Programming using Modern Java
01h 03mBuilding AI powered Applications with Java
01h 01mScaling AI with Java: Building Production-Ready LLM Applications
02h 37mDeveloping Cloud-Native Java AI applications with DJL and LangChain4J
46mAI-powered Java Developers
47mBlazingly Fast GenAI App Development Java and Spring AI