Talking to Objects in Natural Language: Toward Semantic Tools for Exploratory(…)
summary
Proposal builds a working prototype in Squeak/Smalltalk that uses GPT-4o as an exploratory agent which inspects object state, browses source, and executes generated code to answer high‑level functional and epistemic questions. Semantic messages and a chat mode let the agent plan experiments, return code objects, and keep conversation history. It reduces manual probing but sometimes hallucinates, loops endlessly, and can cost up to $60/hour in API bills; demonstrations mostly succeed but reveal reliability and performance limits.
AI-written summary · about
chapters (19)
- 0:00Exploratory programming intro
- 2:08Object interaction challenges
- 3:33Exploratory agent proposal
- 4:23Research question & contributions
- 5:07Exploratory programming goals
- 7:34AI tools limitations
- 8:16Semantic object interfaces
- 9:20Object inspection integration
- 10:22Exploratory agent implementation
- 11:23Demo: Order creation query
- 13:31Demo: Text formatting exploration
- 14:53Extending semantic interfaces
- 16:16Demo: Debugging with queries
- 17:19Approach feasibility & limits
- 18:43Performance optimizations
- 19:23Impact on exploratory experience
- 20:27Future work & agent collaboration
- 21:10Conclusion & contributions
- 22:13Q&A
related talks
01h 01mAI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood
18mTalking Shit About AI Agents
34mProgramming Languages for AI Programing Agents
01h 04mBuilding Expertise Into Generated Conversations
50mAgency
45mWeb Development in the Age of AI
53mUnleash the power of AI: Automate tasks with OpenAI and Semantic Kernel
25mFrom Treatise to Tech: The Development of ChatSOC, an AI Research Assistant