RADOS Object Class Development in C++ Lua

on YouTube ↗30m

" summary

Shows how Ceph object classes let compiled C++ modules or Lua scripts run inside OSDs to perform work like MD5 checksums or ImageMagick thumbnailing so only results cross the network. Demonstrates building libcls_checksum.so, using librados and OMap, and deploying Lua VM in OSDs with per-invoke script shipping and module whitelisting. Cites Skyhook results: pushing predicates to OSDs yields ~6x query speedup for a 10% scan and ~100x for point lookups; erasure-coded pools are not supported.

AI-written summary · about

" chapters (23)

1. 0:00Intro
2. 0:24Ceph architecture
3. 0:55Ceph RBD data management
4. 2:30Outline
5. 3:03Structure of a librados client application
6. 5:01Object classes extend the object API
7. 6:25Invoking a new checksum extension in Ceph
8. 7:26Structure of a C++ object class
9. 10:22RADOS object class API
10. 10:49Build and install object classes
11. 13:02Skyhook: tables are partitioned into objects
12. 13:17Skyhook: pluggable storage engine
13. 14:15Skyhook perf: 10% selectivity query (1B rows, 140GB)
14. 14:59Skyhook perf: scan vs index (1 in 10 rows)
15. 15:56Ceph is the largest source of object classes
16. 16:55Object classes exist in a walled garden
17. 18:01Example: remotely compute md5 hash
18. 19:27A librados client invoking an object class method
19. 20:15Shipping in upstream Ceph
20. 22:41Example: image thumbnail generation with caching
21. 24:50Control fleets of objects with Lua
22. 25:22Additional users
23. 27:03Resources