Panel discussion "NLP"
summary
Panelists report concrete moves: pretrained transformers (BERT, ELMo) drive big gains, HuggingFace’s Transformers library ships models used in benchmarks; model compression via DistilBERT keeps ~95% accuracy with half the parameters; fine‑tuning schedule (multi‑task → similar task → task) and adversarial/data‑augmentation heuristics raise scores on GLUE and SuperGLUE; quantization to 8‑bit and CPU/mobile converts cut latency; multilingual models pool 17+ languages but language‑specific retraining still improves accuracy.
AI-written summary · about
chapters (4)
- 0:00Start
- 15:08Adversarial Example in Nlp
- 17:54Data Augmentation for Nlp
- 35:55The Similarities between Transfer Learning and Multitask Learning
related talks
23mUnveiling the Journey of NLP
32mAn Introduction to Transfer Learning in NLP and HuggingFace
28mFine-Tuning Small Language Models with Python
01h 16mGeneralization in natural language processing
27mDo you know how well your model is doing? Evaluate your LLMs
45mTransfer Learning - Entering a new era in NLP
01h 38mTraining Large Language Models: Practices and Research Questions
25mEasy Access to SOTA NLP Models with Ray and Hugging Face