OTeL/Piped Processing Language: Simplifying Observability Queries With Open...
summary
OpenSearch Piped Processing Language (PPL) provides a Unix-style pipeline for observability queries, claiming Unix-like pipe syntax and compatibility with OpenTelemetry field names so a single PPL query runs across logs, traces, and metrics. Engine translates to Calcite plans and pushes aggregations into OpenSearch; extraction/regex runs at query time. Repo lists 38 commands and 173 functions; demo builds dashboards from discover, shows AI-assisted PPL generation, regex parsing to extract status codes, and a roadmap to run identical PPL on Spark and other engines.
AI-written summary · about
chapters (23)
- 0:00PPR and OpenSearch Observability intro
- 1:01Observability query complexity issues
- 2:49Bucketing and aggregation use case
- 4:11PPR query language explained
- 6:40PPL commands and functions
- 10:14PPR architecture and performance
- 10:36PPL standardization roadmap
- 11:42Resources and open source
- 12:26Live demo by Ashwin
- 13:08Dashboards with PPL in OpenSearch
- 14:10AI mode for query generation
- 17:18Parsing log fields
- 22:33Visualizations with PPL
- 24:46Adding visuals to dashboards
- 25:51Query reusability and auto-source
- 27:38AI mode integration limits
- 29:04Saving queries and history
- 30:32Performance optimization benchmarks
- 31:58Query execution and time ranges
- 34:05Vega visualization improvements
- 34:48Custom scatter plots
- 35:53Default experience and feature flags
- 36:56Q&A and closing
related talks
22mFrom Telemetry to Insight: Building AI-Powered Observability Pipelines with OpenSearch
26mPlug-and-Play Observability: OTel Auto-Instrumentation for LLMs in OpenSearch
33mObservability Pipeline Query Languages: Present and the Future
20mOpAMP in Action: User Configurable Observability Pipelines
24mOne Pane to Rule Them All: Uniting the Prometheus Community with O...
06mSponsored Keynote: OpenSearch - See Everything: Open Observability for Agentic AI
11mLightning Talk: OTel Me How to Build a Data Pipeline for Observability
33mPugs, Poe's and pipelines; An engineering perspective on big-data streams...