Real-World Datomic: An Experience Report
summary
Experience report on using Datomic to index 250 million event datums stored on DynamoDB and served with a transactor that assigns t values and partitioned entity ids. The concrete surprises: queries over EAVT/AEVT/VAET alone miss a 100 ms SLA, so the system builds in-memory summaries by scanning indexes with seek-datums and fabricating start ids with ntid, partitioning by event type, and caching day-by-day counts. The lesson is which indexes, cardinalities, and ref reverse indexes actually make or break real performance.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:43The Problem • Provide a summary of sales funnel events for Room Key • Users move through the website impression - hotel details - lead - booking . Want per-day event counts by hotel & event type
- 3:55Datomic's Architecture
- 5:53Datomic's Data Model
- 10:23Capturing Events - Schema Type Qualifiers
- 13:40Capturing Events - Example
- 15:50Capturing Events - Refs
- 16:49Storing and Retrieving Data in Datomic • One or more EAV tuples submitted to the transactor All are given the same value . Datoms written transactionally to the log . All datoms also stored redundantly in the indexes
- 21:53EAVT and AEVT
- 23:59Capturing Events - EAVT
- 25:09AVET and VAET
- 26:59Index Storage and Retrieval . Datoms are stored in compressed chunks called segments • Segments are stored as a tree • Segments are immutable once written • Each transaction conceptually creates a new root
- 28:49Indexing - Append
- 32:29Anatomy of an Entity ID
- 34:55A Change In Approach
- 37:32The Code
related talks
37m11 insights after 11 years with the functional database Datomic
40mThe power of Datomic database
51mDatomic
37m11 insights after 11 years with the functional database Datomic
41mExploring four hidden superpowers of Datomic
46mThe Datomic Architecture and Data Model
40mDatomic: The Most Innovative DB You've Never Heard Of
26m"Operating Datomic at Scale"