CodeMend: Assisting Interactive Programming with Bimodal Embedding
summary
CodeMend uses a neural bimodal embedding to map natural‑language goals plus local code context to concrete code edits, implemented for Python and matplotlib and trained on ~50,000 code examples and ~90 million words. It flattens ASTs into code‑ngrams, surfaces function → parameter → value choices, and auto-inserts or modifies lines. Model rank puts correct suggestions around 4th on average; evaluation uses 361 queries from 45 image pairs and a 20‑participant lab study (108s vs 134s trend), with 70% of users saying it helps.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:25Programming is not so easy
- 5:20What you've seen in the video
- 5:58Traditional Solutions
- 6:21Three Scenarios
- 6:35Deep Learning is a Good Fit
- 7:15How does this work?
- 8:03Opening the black box
- 9:05Map Code to Neural Net
- 11:35Interface Design - Inspiration
- 13:04How well does it work?
- 13:22Experiment 1: How well does the model work?
- 14:21Experiment 1: Example Model Output
- 14:57Experiment 1: Results
- 15:21Experiment 2. How well does the system work?
- 16:36Experiment 2 - Results
- 17:39Future Directions
- 18:15Summary
related talks
46mCode + AI: Will Robots Take Our Coding Jobs? ML Applied to Programming
11mGen AI-Assisted Code Authoring
37mMachine Learning on Source Code
43mMachine Learning for Developer Productivity
19mCodeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming
15mNatural Language-Guided Programming
24mNatural Language Coding Revolutionising Programming with GenAI
27mTeaching a Machine to Code