Mining Noisy Transaction Data with Neural Nets
summary
Company transaction logs total roughly 1.5 TB and 100M rows; 22,000 rows get hand-labeled in a labeling party to create ground truth. Text tokens are vectorized with skip-gram word2vec / GloVe, centroids per transaction combine with normalized amounts, and classifiers run on Spark/Scala using Deeplearning4j, logistic regression, boosted trees, random forest and a simple MLP. Results: logistic 86%, boosting/random forest 88%, single‑hidden‑layer MLP 94%. Distributed NN training works but shows instability and synchronization overhead.
AI-written summary · about
chapters (12)
- 0:00Introduction and goals
- 2:06Company overview and mission
- 4:36Transaction data challenges
- 8:06Potential use cases for data
- 11:19Why traditional NLP failed
- 16:26Word vectorization methods
- 22:12Preprocessing and vectorization
- 23:16Modeling approach and results
- 26:27Distributed neural networks
- 28:12Workflow and evaluation
- 29:17Next steps and future models
- 30:52Q&A session
related talks
24mRetail Transactions at scale for AI - From PoC to production
28mBeyond the Balance Sheet: Transactional Data as a Window to Company Health
27mLog messages processing using NLP tools
24mMulti-tasking fraud detection From Trees to MLPs
14mCloud-Native AI for Fraud Prevention with Bayesian Networks
26mLarge Scale Graph Processing & Machine Learning Algorithms
33mWTF am I doing? An introduction to NLP and ANN's
48mHunting Stealth Adversaries with Graphs & AI