POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Ima...
summary
POET is a real-time tool that uses prompt inversion (soft prompt tuning + CLIP similarity) to extract homogeneous prompt embeddings, an LLM to categorize and expand those dimensions, and preference conditioning on Likert and selection feedback to personalize outputs. A live prototype generates expanded prompts for a base diffusion model and, in a within-subjects study with 28 participants across four domains, produces perceptibly more diverse images, reaches user satisfaction in fewer rounds, and shows significant diversity gains for fictional character and interior design scenarios.
AI-written summary · about
chapters (7)
- 0:00Introduction to Poet
- 1:55Limitations of text-to-image models
- 2:56How Poet works
- 5:02Diversifying generated images
- 6:04Personalizing outputs
- 7:47User study results
- 9:52Summary and conclusion
related talks
29mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
13mPrompt Engineering Simplified
29mSupercharge Your Productivity: Mastering Prompt Engineering
30mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
01h 08mIn Prompts We Trust
44mSupercharge Your Productivity Mastering Prompt Engineering
30mIs Prompt Engineering Dead?
01h 04mIn Prompts We Trust