Best Practice: DNS Failure Observability and Diagnosis in Kubernetes
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Friday, 2021-12-10· at 11:20· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Covers common DNS lookup failure scenarios in Kubernetes and their root causes (mostly container networking and misconfiguration). Describes CoreDNS observability using plugins (log, errors, trace, dump, metrics), diagnosing CoreDNS via the dnstap protocol with a context-based analyzer as a lighter-weight alternative to tcpdump, and using eBPF on the client side to diagnose DNS failures without impacting DNS servers.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:19DNS in Kubernetes
- 1:07Under the hood
- 1:50Surprises
- 3:21DNS becomes fragile
- 4:41Resource limits
- 6:00Conflicts
- 7:22Bugs
- 8:04Built-In Observabilities
- 8:24Loggings - log
- 9:21Loggings - dump
- 9:49Loggings - debug
- 11:49Loggings - dnstap - how to analyze
- 13:56Tracing - trace
- 14:34Metrics - prometheus
- 15:52BPF-based Tools for diagnosis
- 17:42Diagnostic Procedure
- 20:50Conclusion
related talks
39mObservability to Resolution: The Journey Through a Production K8s Incident
35mKubernetes, Lessons Learnt
22mWhy is it always DNS, TLS, and bad configs?
29m“Are These Things Talking to Each Other?” - Observing Kubernetes Networking...
11mKubernetes Observability
35mCrashLoopBackoff, Pending, FailedMount and Friends: Debugging Common Kubernetes Cluster
36mConfidence with Chaos for Your Kubernetes Observability
39m15 KUBERNETES FAILURE POINTS YOU SHOULD WATCH