byte2vec: a flexible embedding model constructed from bytes
summary
byte2vec builds word embeddings from raw bytes rather than tokens, producing language‑independent vectors that handle out‑of‑vocabulary and morphologically rich languages. Model architecture turns byte sequences (≈256 vocab) into byte vectors, applies convolutions + max‑pool + highway layers, then projects to word vectors and trains with a skip‑gram objective. Evaluated on Twitter sentiment and aspect‑based sentiment tasks, it improves handling of noisy multilingual data; code is being released and a paper submitted.
AI-written summary · about
chapters (8)
- 0:00Introduction and company overview
- 2:29Products and APIs explained
- 8:34Sentiment analysis variations
- 11:31Limitations of word embeddings
- 15:09Introducing byte2vec model
- 16:55Evolution of sentiment analysis
- 18:18Language independence in models
- 19:00Q&A session
related talks
28mEmbeddings
40mBuilding an Open-source Framework for Generating Embedding Vectors
28m"Embeddings: How Computers Learned to Read"
29mDataEngBytes 2023 - SYD-T2-01 - Rene Essomba
26mImplementing a Search Index
01h 03mWhat comes after ChatGPT? Vector Databases - the Simple and powerful future of ML?
01h 03mWord Embedding For Fun and Profit
45mA gentle introduction to Vector Databases