🤖 Transformer-based NLP Pipelines with SpaCy v3
summary
Shows how spaCy v3 plugs transformer models (BERT/Roberta family) into pipelines and how replacing separate task encoders with a shared Transformer listener yields one encoder for NER + entity linking, cutting duplicate parameters and running the encoder once at inference. Demonstrates concrete CLI workflow: spacy convert / init config / spacy train, how to format data, generate config that chooses shared vs independent embeddings, and train on GPU then deploy the resulting pipeline on CPU. Notes multitask can improve efficiency and generalization but can also fail if tasks don’t share structure.
AI-written summary · about
chapters (10)
- 0:00Introduction to talk
- 1:05Transformers overview
- 3:37Language modeling with Transformers
- 5:47Single vs multitask learning
- 8:16What is SpaCy v3
- 10:23Transformer components in SpaCy
- 12:48Building pipelines with Transformers
- 14:35SpaCy configuration system
- 15:58Practical implementation demo
- 19:54Summary and resources
related talks
01h 27mTips, Tricks and Topics in Text Analysis
01h 14mIntroduction to NLP
40mDesigning spaCy: Industrial-strength NLP
30mKick-Start your Understanding of Machine Learning with Python
35mPandas, Pipelines, and Custom Transformers
25mInventing Curriculum using Python and spaCy
24mA Journey through Generative AI's Frontier Python Libraries
01h 39mNatural Language Processing (NLP) in Python - From Zero to Hero