Towards controllable language generation
summary
Deep learning models regain controllability through concrete methods: using noise injection plus self‑training on the E2E data yields a new seq2seq “faithful speaker” that cuts semantic error rate (a rule‑based parser variant hits 0%) even if BLEU/Rouge fall; 30% of abstractive summaries are initially unfaithful, so a QA‑based metric QUALS (55× faster than prior QAGS) paired with contrastive learning raises QAGS by ~4 points on XSUM and ~2 on CNN/DailyMail; and alignment training with planners produces controllable word order, driving semantic errors toward zero and achieving order accuracy above human baselines.
AI-written summary · about
chapters (5)
- 0:00Introduction to controllable generation
- 2:16Faithful generation from structured input
- 13:29Faithfulness in abstractive summarization
- 20:09Controlling word order in generation
- 25:38Summary and future directions
related talks
01h 00mHuman and machine inductive biases for compositional linguistic generalization
01h 05mControlling distribution shifts in language models: a data-centric approach.
39mTackling the Limits of Deep Learning
01h 01mControllable and Creative Natural Language Generation
53mGenerative AI: Overview and use cases
29mLies, damned lies and large language models
01h 31mNatural Language Understanding: Foundations and State-of-the-Art
41mTeaching Exotic Programming Languages to Large Language Models