Full-Stack AI in Java: From Cassandra to CarPlay - A Production SaaS Story
summary
SkillsPilot is a production Java SaaS for sensitive HR data that runs end-to-end in Java and achieves 99.9% uptime, sub-200ms latency and 10,000+ concurrent capacity after 18 months. The stack uses Cassandra + pgvector for multi-tenant storage and embeddings, Langchain4j orchestrating multiple LLMs for parallel RAG, Spring (core) for business logic, WebForJ/Flutter frontends and CarPlay/Android Auto clients. Cost measures include semantic caching, model routing and spot instances; deployment supports on‑prem with per‑tenant encryption.
AI-written summary · about
chapters (21)
- 0:00Full-stack AI in Java
- 3:32Skills Pilot and HR use cases
- 5:20Architecture and tech stack
- 8:46Java over Python for AI
- 13:59Java consistency in teams
- 14:19Cassandra database selection
- 18:58Deployment flexibility
- 21:48Prompt management
- 27:25Switching AI models
- 31:01Data privacy and security
- 32:08LLM cost comparison
- 35:16Cost-saving strategies
- 36:20EU AI Act compliance
- 38:05Deployment and GraalVM
- 40:09Frontend choices
- 41:54Challenges with car play and app store certification
- 42:57Lessons learned
- 43:19What we would do differently in hindsight
- 45:02Platform constraints
- 47:13AI access control
- 48:40Key takeaways
related talks
45mBuilding Intelligent Enterprises: AI-Powered Java with LangChain4j & CDI
26mJava’s GenAI Era: Getting Spring AI and Neo4j Applications to Production
01h 03mBuilding AI powered Applications with Java
49mSpring AI: Seamlessly Integrating AI into Your Enterprise Java Applications
47mBlazingly Fast GenAI App Development Java and Spring AI
49mPractical AI in Java
25mAI for Java Developers
52mBuild Smarter Knowledge Management Systems with AI Technology