Generative Trigger-Action Programming with Ply
summary
Ply uses LLMs and a vision-language model to generate composable trigger-action code by building incremental layers: a layer abstraction that decomposes sensors and actuators, automatically produced generated visualizations at each layer boundary, and generated configurable parameters with UI controls. Demo connects a camera and smart outlet to detect rice-cooker states and blink a lamp; microphone transcription and slide-text examples show multi-dependency layers. Technical tests and a first-use study with seven participants show visibility aids help debugging but users still often accept monolithic, end-to-end code.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 2:51Ply approach and design
- 3:32Live demo of Ply
- 6:00Layer reuse and customization
- 7:03User study insights
- 8:08Future directions and call to action
- 8:31Q&A session
related talks
41m"Natural Interfaces" and "Natural Language Programming"
01h 15m10x Development: LLMs For the working Programmer
19mtldraw.computer
12mNavigating the World of AI Development Without Going Insane
01h 00mFrom LLM Theory to Practical Agentic Implementations
35mBuilding LLM-based Agents
26mLanguage Model Agents Enable Semi-Formal Programming
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI