AI vs. Karen: Real-Time Customer Sentiment Detection at Scale
summary
Real-time pipeline ingests customer messages via Apache Kafka, runs sentiment in Flink using OpenAI (GPT‑4.1 Nano) then stores results in Apache Pinot and dashboards in Superset; 81% accuracy with GPT‑4.1 Nano but calls cause scaling lag (consumer lag hits 3.5k (~5 minutes)), so local encoder models from Hugging Face run inference on-node orders of magnitude faster but miss extreme “Karen” cases until a ModernBERT variant is fine‑tuned (via Hugging Face AutoTrain for ~$2) and restores recall while keeping subsecond throughput.
AI-written summary · about
chapters (9)
- 0:00Welcome
- 2:52Real-time AI data pipeline
- 7:03Sentiment analysis challenges
- 11:17Real-time streaming architecture
- 27:30Live demo
- 39:51Problems with large language models
- 42:18Small language models
- 55:35Fine-tuning for Karen
- 1:02:57Final demo
related talks
47mAI vs. Karen: Real-Time Customer Sentiment Detection at Scale
43mSentiment Analysis in Action: Building Your Real-time Pipeline
25mReal-Time Fraud Detection at Scale
41mAdding Generative AI to Real Time Streaming Pipelines
58mAI-Enhanced Big Data: Integrating Private LLMs and Vector Databases
29mBuilding Resilient AI Platforms: Financial Fraud Detection
21mAI as an API
20mHuman-Centered AI for Sustainable Development