Feature Flags and GenAI parameterization and experimentation
summary
Shows how feature flags act as a remote parameter store so AI parameters change without redeploys, with concrete code examples calling Split/Harness via getTreatmentWithConfig to toggle models and JSON params; demos include use stable diffusion 1.5 vs 1.4 and GPT/RAG configs (temperature, tokens, chunks), 50/50 and percent rollouts for A/B, and cost/performance tradeoffs (e.g. 512→384 image size). Covers experiment attribution challenges with concurrent flags, feature‑flag debt, CI/CD flag automation, multi‑arm bandits, and tooling like Uber Piranha.
AI-written summary · about
chapters (13)
- 0:00🚀 Build Stuff Conference – Get Your Tickets!
- 0:21Introduction: Feature Flags and Generative AI
- 3:15The Role of Feature Flags in Software Development
- 6:40Types of Feature Flags: Release Flags vs Experiment Flags
- 9:10The Concept of Parameterization in AI Models
- 13:00How Feature Flags Can Enable Dynamic AI Experimentation
- 16:20Code Example: Managing AI Parameters with Feature Flags
- 21:00Benefits of Testing AI Models in Production
- 24:00AI-Driven Experimentation and A/B Testing
- 28:00Managing AI Performance with Multiple Feature Flags
- 31:45Best Practices for Managing Feature Flag Debt
- 35:00The Future of AI Experimentation and Flag Automation
- 39:00Conclusion: Integrating Feature Flags with AI for Smarter Software
related talks
19mAgents Need Feature Flags
27mAI Ships Fast and Breaks Things — Feature Flags Keep It Controlled
14mHow leading AI companies 10x experimentation velocity with Statsig
48mMastering the AI Shift: Adaptive Strategies for Software Evolution
45m9 Lessons Learned from Deploying GenAI at Scale
51mPlatform Engineering in the age of Generative AI
33mOperationalizing GenAI Applications with DevOps Practices
01h 07m📚 Building Portable Large Models on Heterogeneous AI Accelerators Wit...