Real-time Transaction Categorization w/ Bayesian Feedback Loop
summary
ING built a production system to categorize customers’ bank transactions in real time using a batch/streaming architecture: PySpark batch jobs collect implicit and explicit customer feedback and use Bayesian inference and bandit strategies to update category probabilities, while an Apache Flink real-time service applies the batch-derived models via PMML to label transactions as they occur. The talk explains the tooling and constraints, how the initial algorithm was bootstrapped to reach acceptable quality, and the Bayesian-inspired feedback loop that lets the categorizer self-learn from customer responses without relying on deep learning.
AI-written summary · about
chapters (8)
- 0:00Real-time transaction categorization
- 3:00Why we built this
- 8:33Kafka and Flink architecture
- 14:01Bayesian feedback loop
- 22:28Balancing feedback types
- 27:31Future plans
- 31:05Monitoring engagement
- 37:03Handling feedback
related talks
33mSmart Banking - Real Time Driven
37mRealtime Bayesian A-B testing with Spark Streaming
47mAI vs. Karen: Real-Time Customer Sentiment Detection at Scale
24mUnder Pressure: Applying ML in Real Time
16mOperationalizing AI in Finance: Real-Time Analytics
01h 06mAI vs. Karen: Real-Time Customer Sentiment Detection at Scale
25mReal-Time Fraud Detection at Scale
13mTransforming Financial Services with Real-Time Analytics