An Introduction to Edge Extensions, Client API and Data Explorer
summary
Shows three concrete StreamPipes additions: a Data Explorer for persisted traces with charting (defaults to last 100 events and supports aggregates and histograms), a Client API exposing REST plus a Java client for actions like list and start pipelines, subscribe to intermediate processors and fetch historical data via API keys, and Edge Extensions (node controller that self‑registers via docker compose, handles ARM vs x86 images, offers buffering, preemption and an event relay with live migration). Demos run a high‑frequency sensor reduced to 2000 ms and start/stop pipelines successfully.
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:10Apache Stream
- 6:25Data Explorer
- 12:50Client API
- 13:52External Applications
- 15:00Client API Architecture
- 15:21How does it work
- 15:47Supportive features
- 16:45Client demo
- 20:42Edge Extensions
- 21:40Stream Pipes Architecture
- 24:59Node Controller
- 27:26Docker Compose
- 29:05Cluster Overview
- 30:52Pipeline Overview
- 31:19Rate Limiting Component
- 32:21Rate Limiting Visualization
- 32:41Advanced Deployment Settings
- 36:30Live Migration
- 38:49Summary
related talks
50mDeveloping (on) for the Edge
33mFlink for Everyone: Self-Service Data Analytics with StreamPipes
31mWhat is Edge Native?
26mSimplifying Data Analytics Using RudderStack Open Source Data Pipelines
40mDeep Dive: KubeEdge
15mAzure IoT from the edge to ADX
21mDesigning Modular Data Pipeline
35mEvent Streaming Architecture for Industry 4.0