AI Workloads and NGINX: Navigating Agentic Traffic Flows
summary
NGINX adds agentic observability in NGINX Plus R37 using the JavaScript module to unpack MCP flows, count per‑tool bandwidth and errors, and surface which agents and tools are active. It introduces an option to pause streaming and fully inspect requests so routing decisions use MCP metadata or prompt content. It highlights the need to scrape GPU inference metrics multiple times per second and points to the Gateway API endpoint picker for load aware routing.
AI-written summary · about
chapters (9)
- 0:00AI workloads with Nginx
- 1:47Agentic observability in Nginx Plus R37
- 3:09Enterprise challenges with AI agents
- 5:57Inspecting AI traffic for routing
- 8:01Token cost and sentiment routing
- 8:44Load balancing for AI workloads
- 10:53Monitoring GPU cache performance
- 11:34Kubernetes Gateway API for AI routing
- 12:36F5 in AI infrastructure
