Plugin-based software design with Ruby and RubyGems
summary
Plugin architecture described uses a thin core with most features as plugins: a C core collects data and Ruby plugins provide I/O and transforms, exemplified by Fluentd and Embulk. Uses RubyGems plugin loader to auto-discover installs, handles hundreds of gigabytes per day in production, and exposes a 300+ plugins ecosystem. Embulk runs a Java core with JRuby plugin loading and per-plugin classloader hacks to allow differing dependency versions; classloader isolation works for Java but Ruby VM gem-version conflicts remain unsolved.
AI-written summary · about
chapters (20)
- 0:00Intro
- 2:56Benefits of Plugin Architecture
- 4:48Traditional Extensible Software Architecture
- 5:49Plugin-based software architecture
- 8:38Dependency Injection (Java)
- 9:47Dependency Injection (Ruby)
- 11:18Dynamic Plugin Loader
- 12:23Combination
- 12:54Plugin Architecture Design Patterns
- 16:21Example: Simple forwarding
- 18:28Fluentd's Plugin Architecture
- 22:46Use case 1: Syne MySQL to Elasticsearch
- 26:06Embulk's Plugin Architecture
- 28:37Plugin Package Structure
- 29:36Embulk Plugin Load Sequence
- 32:42Plugin Version Conflicts
- 33:17Version conflicts in a Ruby Runtime
- 33:44Version conflicts in Fluentd
- 34:11Multiple JRuby Runtime?
- 35:10What's Next?
related talks
54mBuilding and Managing a Ruby Infrastructure
39mBuilding Generic Software
26mBuilding Stream Processing Applications with Ruby & Meroxa
42mExtending Gems - Patterns and Anti-Patterns of Pluggable Gems
28mFluentd: Data Streams in Ruby World
35mExploring Merb
31mBuild the Unified Logging Layer with Fluentd and Ruby
42mRubyGems to All JVM Languages