Machine Learning on Source Code
summary
Claims that identifier embeddings capture code naturalness by mapping millions of names to dense vectors, trains them from an explicit PMI co-occurrence matrix extracted from code contexts, and scales via sharded SGD using the Swivl model with GPU and distributed TensorFlow. Pipeline processes ~200k repositories to build a 1M×1M vocabulary, finds nearest neighbors that recover synonyms and common typos, and powers autocomplete by minimizing summed embedding distance — a practical system that demonstrably improves suggestions.
AI-written summary · about
chapters (5)
related talks
37mMachine Learning on Source Code
25mMachine Learning for source code analysis
28mDelivering AI on Code: Live Demo of source{d}
43mMachine Learning for Developer Productivity
28m"Embeddings: How Computers Learned to Read"
27mMachine Learning for Advanced AI Search
44mA Theoretical Approach to Semantic Coding and Hashing
31mTaking AI/ML to the next level: Text!