Analyzing Operational Data at Scale Using ML at Intuit
Talkat KubeCon + CloudNativeCon North America 2020 Virtual · on Friday, 2020-11-20· at 17:55· Intrado Virtual Event Platform,·
summary
Intuit runs ~2000 services on Kubernetes and builds an Operational Data Lake that ingests events and metrics via stream processing into a schematized warehouse, using compact data structures, an in-memory cache for hot queries and cold storage for long term. Simple ML models flag anomalies and rank likely causal service detection candidates so operators see impacted services and a prioritized root cause within seconds, demonstrated end-to-end on production-like traffic.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 3:06Data collection lifecycle
- 10:13Operational data challenges
- 16:15Real-time data insights
- 20:13ML models for data analysis
- 22:57Conclusion and Q&A
related talks
28mAccelerating AI at Scale with Data
24mInference on Streaming Data at Scale at Intuit
50mIntuit Journey To Unified Observability at Scale: Challenges, Be...
19mHow Intuit Streamlined AI/ML Inference Workflows on K8s
30mBetter data quality with ML & visualization
32mMaking Models Work: How an optimized ML lifecycle can drive better ROI
10mOptimizing Machine Learning (ML Ops) in Financial Services with Kubernetes and Cloud Technologies.
15mHow AI and ML unlock insights for financial services teams