Controlling App Screens Remotely with RFW and Gemini AI
summary
A pipeline uses Google Generative AI (Gemini Pro via the google_generative_ai package) to return RFW JSON that the app interprets into remote Flutter widgets, updating only the RFW subtree. The system sends a 15,000-token prompt (runs Gemini with a 20,000-token response limit and temperature 0.1), parses a sentinel RFW execute token, strips formatting, and applies the JSON to a ValueListenable RFW string to rebuild widgets and fire string-named callbacks; demos mostly work but hit API throttling and parsing quirks.
AI-written summary · about
chapters (19)
- 0:00Welcome
- 1:48Talk overview
- 2:30What is RFW
- 3:51Google Generative AI
- 6:21Retrieval augmented generation
- 7:25JSON for AI versatility
- 9:14Live demo prep
- 19:08Demo setup
- 21:12AI prompt building
- 23:00Prompt engineering
- 25:27RFW callbacks
- 28:20Sending to Gemini
- 30:55Updating UI
- 33:25Handling errors
- 35:12Returning to the live demo after technical issues
- 38:00Interactive buttons
- 40:23AI code conversion
- 43:25Testing Flutter code
- 47:50Summary
related talks
36mEmpowering Your Flutter Apps with AI
38mGenerative UIs: Using LLMs to Create Flutter Screens at Runtime
38mEnhance the customer experience
14mPrompt to production: Building an AI app with Flutter
01h 11mBuild Your Own AI Sidekick with Flutter
47mAgentive Experiences With Flutter: What, How, and Why Bother?
46mLessons Learned Building A GenAI Powered App
01h 55mPrompt to Pipeline: Building with Google's Gen Media Stack