package2vec: getting to know PyPI packages with ML
summary
Describes applying state-of-the-art NLP and Graph Neural Network methods to create learned vector embeddings of Python packages on PyPI. Explains using those embeddings to surface package similarity and topics and discusses using them for developer-facing package recommendation features.
AI-written summary · about
chapters (24)
- 0:00Intro
- 2:13The Goal
- 2:50Representation Learning
- 5:06What Makes up a Python Package?
- 7:50Collecting the Data
- 9:39At a Glance
- 10:52Vector Spaces for Similarity Metrics
- 12:16TF-IDF Intuition
- 13:51TF-IDF - What it tells is
- 15:18Advantages and Disadvantages of TF-IDF
- 17:29The Distributional Hypothesis
- 17:50Interesting Properties of Word2Vec
- 20:32Word2Vec Negative Sampling Intuition
- 22:39Advantages and Disadvantages of Doc2Vec
- 23:36Ash API Demo
- 27:10Latent Dirichlet Allocation
- 30:40Advantages and Disadvantages of ETM
- 32:32Training Examples
- 33:40Community Graph Structures
- 35:33Advantages and Disadvantages of DeepWalk
- 36:29A Joint Approach
- 38:44How to Evaluate?
- 39:33Evaluating with Logistic Regression
- 42:16In Conclusion
related talks
38mmusic2vec: A smart music recommendation and discovery
28m"Embeddings: How Computers Learned to Read"
16mMachine Learning on Source Code
37mLike2Vec with neural network embeddings
49mlda2vec
50mVector Ops: run vector embedding-powered apps in prod
31mEmbeddings... Embeddings Everywhere
21mGraph Representation Learning with Deep Embedding Approach