Lightning Talk: Entity Store: A FoundationDB Layer for Versioned...

on YouTube ↗05m

" summary

A FoundationDB-backed Python library called Entity Store provides versioned ML metadata with automatic versioning (immutable-field changes create new versions), unique version IDs derived from SHA-1 of primary-key plus parent/version/auth context, and transactional concurrency. Field-level read/write permissions are recorded per cell and indexed fields and large blobs are supported. Provenance is stored as labelled directed multigraphs and the layer exposes FoundationDB layer semantics and cell-based security so clients get versioning, lineage, and auth for free.

AI-written summary · about

" chapters (10)

1. 0:00Start
2. 0:28A FoundationDB Layer
3. 0:52Why does metadata matter for machine learning?
4. 1:08Pipelines start with lots of datasets
5. 1:20Data wrangling transforms datasets
6. 1:42Training produces models
7. 2:05Experimentation iterates over models
8. 2:56Entity Store
9. 3:13Authorization
10. 3:23Versioning

" related talks

• 25mBuild Anything with FoundationDBRyan Worl · FoundationDB Summit 2019
• 32mMagical Layers - The Layered Entity ConceptChristian Kuemmel · JCON-ONLINE 2021
• 06mMeta-Data Caching and Record Store ScalabilityAlec Grieser · FoundationDB Summit 2019
• 18mFoundationDB Record Layer: Open Source Structured Storage on FoundationDBNicholas Schiefer · FoundationDB Summit 2019
• 46mFoundationDB: The Best-Kept Secret of New Distributed Architectures!S.LE ROUX, P.ZEMB · Devoxx France 2023
• 24mHow We Saved 5x Migrating from Aurora to FoundationDBRay Jenkins, Rick Branson · FoundationDB Summit 2019
• 50mBuilding the Next-Generation Backend on FoundationDBOleg Illarionov · HighLoad++ Moscow 2018
• 20mUsing FoundationDB and the FDB Record Layer to Build CloudKitScott Gray · FoundationDB Summit 2019