Monitoring Multiple Cloud Providers Made Easy

on YouTube ↗53m

" summary

Elasticsearch centralizes logs and metrics from AWS, Azure and GCP using Filebeat, Metricbeat and the serverless Functionbeat. Metricbeat pulls 16 metric sets from AWS (incurring CloudWatch API charges), Filebeat reads S3/Event Hub/Stackdriver sinks, and Functionbeat deploys as a Lambda that watches the CloudWatch log group "Apache access logs" (uploads a CloudFormation package to S3, creates Lambda and IAM roles) and ships raw log messages to Elasticsearch (data arrives in ~2–3 minutes; use ingest pipelines to split message fields).

AI-written summary · about

" chapters (8)

1. 0:00Introduction
2. 1:47Challenges in multi-cloud monitoring
3. 5:02Monitoring AWS logs and metrics
4. 16:32Monitoring Azure logs and metrics
5. 19:02Monitoring GCP logs and metrics
6. 23:23Adding APM to observability
7. 28:49Demo: Function Beat for serverless monitoring
8. 48:08Q&A and closing remarks

" related talks

• 34mMonitor Kubernetes cluster with ElasticAshish Tiwari · KCD Mumbai 2023
• 49mMonitor Your Microservices with Logs, Metrics, Pings and TracesDavid Pilato · Voxxed Days Microservices 2018
• 30mUsing Open Source Observability with LambdaMike Elsmore · GOTOpia Europe 2020
• 34mObservability in Serverless ApplicationOzioma Uzoegwu · Conf42 Site Reliability Engineering 2021
• 35mFull Stack Visibility with Elastic: Logs, Metrics and TracesCarlos Pérez-Aradros · KubeCon + CloudNativeCon North America 2017
• 44mThe Holy Trinity of Observability: Logs, Metrics, and TracesPhilipp Krenn · Oredev 2017
• 31m360° Monitoring of Your ServicesPhilipp Krenn · CloudNativeCon + KubeCon Europe 2017
• 43mPractical Application Logging and MonitoringLaurynas Tretrjakovas · DevOps Pro Vilnius 2016