AI/ML Data Pipeline Processing with Go Microservices Based...
summary
Containerized Go microservices orchestrate automated image transfer, preprocessing and inferencing using EdgeX Foundry with Redis for job/task state, Project AIR for pipeline composition and Intel OpenVINO for optimized edge inference, communicating over an MQTT message bus. The system runs file watchers on Windows OEM devices and a Linux gateway that executes pipelines and returns outputs; it works as a proof of concept but exposed deployment pains: Windows installer vs zip, flaky end-to-end timing/idempotency, added wait-for-TCP logic, migration away from Project AIR after its shelving, and planned open source release on open.intel.com.
AI-written summary · about
chapters (15)
- 0:00Intro
- 1:42What is IoT
- 5:00Project Objectives
- 7:24Architecture
- 11:32Architecture Diagram
- 17:33UI Features
- 24:14Challenges Learnings
- 26:03Deployment Steps
- 27:52Service Weight Strategy
- 28:31Go Application Code
- 30:00Observability
- 31:46Testing Issues
- 33:26Project Shift
- 34:08Project Update
- 35:01Questions
related talks
43mGo Microservices for ML at the Edge
15mAI-Ready Microservice Architecture
21mHarnessing Microservices for Scalable, Fault-Tolerant ML Systems
14mManaging decisions with Amazon Kinesis for better outcomes
19mEdgeX Foundry - An Open Source Platform for Edge/IoT Solutions
20mAccelerating innovation using tiered AI Architecture
31mIoT Action at the Edge– EdgeX Foundry 1.0 Released!
26mThe Journey to Microservice Adoption