Improve classification accuracy for customer contact transcriptions
summary
KPN processes ~7 million calls and ~700k chats yearly and starts from 180k labeled calls and 40k chats to compare Naive Bayes, CNN with word embeddings (and LSTM/hybrid) and unsupervised methods (LDA, doc2vec+kmeans). Results: Amazon-review CNN hits ~88%/65% (level1/2), chat CNN ~65%/50%, call transcripts ~51%/40%; estimated human agreement ~61–71% and 52–69%. Unsupervised clustering proves unstable; ontology tagging (vector similarity or word-count) reduces labeling cost but yields lower F1. CNN works for chat when lots of labels exist; ASR noise and multi-topic calls prevent good call performance. CNN needs ~50k labeled examples and ontology can be validated with ~5k
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:05Contact volumes and challenges
- 2:34Problems with categories
- 3:38Supervised learning and baselines
- 5:43Single-label limitations
- 6:07CNN for multi-label
- 7:55Performance vs humans
- 9:00Call transcript challenges
- 10:25Unsupervised learning exploration
- 11:27LDA topic modeling results
- 13:58Doc2Vec and K-means attempts
- 16:01Unsupervised limitations
- 17:06Building ontology
- 19:16Ontology tagging
- 22:08Evaluating ontology
- 23:34Ontology results and challenges
- 25:18Ontology vs supervised
- 26:01Recommended approach
- 26:41Future market categories
- 27:22Q&A
related talks
33mImpact of NLP in AI-powered Bank
28mEnhance speech recognition accuracy using custom language models
31mDeep Learning Techniques for Natural Language Processing
39mSentiment Analysis, its techniques and applications
38mLeverage Reviews Data for Multi Label Topics Classification
32mHarnessing Large Language Models for Enhanced Customer Experience
19mRushin Shah, Engineering Manager, Facebook
45mDeep Learning for Natural Language Processing