Infusing Trusted AI using Machine Learning Payload Logging on Kubernetes
summary
KFServing on Kubernetes logs every request and response as CloudEvents to an event broker (Kafka) and a connector syncs those events into MySQL; AIF360 runs hourly fairness metrics and pushes them to Prometheus, visualized in Grafana. Demo sends batches of 10 payloads, the DB grows to ~86 rows, and disparate impact settles near 0.54 versus the ideal 0.8–1.2, showing the pipeline works and surfaces measurable bias for remediation.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:34Production Model Serving? How hard could it be?
- 2:22KNative
- 3:36KF Serving: Default and Canary Configurations
- 4:30Supported Frameworks, Components and Storage Subsystems
- 5:32Inference Service Control Plane
- 6:46KFServing Deployment View
- 7:54KF Serving Examples
- 8:32Model Serving is accomplished. Can the predictions be trusted?
- 9:24Production ML Architecture
- 12:09Payload Logging Architecture Examples
- 13:08Linux Foundation Al & Data
- 13:48Trusted Al Lifecycle through Open Source
- 16:40Al needs to explain its decisions!
- 18:39Bias in Al: Criminal Justice System
- 20:25Adversarial Robustness
- 21:04Al Explainability 360
- 21:58Al Fairness 360
- 22:35LFAI Trusted Al Projects with Kubeflow Serving
- 26:28Demo Flow
related talks
22mAI in Healthcare: Bias, Privacy, and Trust in Kube-Native Systems
25mSecuring AI Pipelines: Real-World Attacks on Kubernetes-Based AI Infrastructure
38mAI made easy with Flink + AIFlow
30mStand up for ethical AI! How to detect and mitigate AI bias using Kubeflow
42mAgentic AI with AWS Streaming
42mProcessing Millions of Events per Second with Apache Kafka on Kubernetes
35mBuilding a Medical AI with Kubernetes and Kubeflow
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes