Mria - an eventually consistent Mnesia

on YouTube ↗26m

" summary

Mria adds eventual consistency to Mnesia and replaces full synchronous replication with a hybrid core/replicant design that shifts writes to a few core nodes while replicants subscribe to an intercepted transaction log. It reports running EMQX clusters of 20+ nodes, 100 million simultaneous clients, 50 million unique topics and keeps reads local for low latency. Testing combines lock-free Erlang and Concuerror model checking with fault injection and trace-based backfill verification; a small OTP hook intercepts post-commit transactions and bootstrap replay makes restarted replicas converge.

AI-written summary · about

" chapters (11)

1. 0:00Mria project intro
2. 1:49Mnesia fits EMQX use case
3. 2:51MQTT basics and challenges
4. 5:44Mnesia write performance
5. 6:50Mnesia replication protocol
6. 9:40Mnesia commit flaws
7. 12:07Mria hybrid architecture
8. 16:22Consistency vs performance
9. 17:25Bootstrapping replicants
10. 20:56Replicant state machine
11. 22:01Key takeaways

" related talks

• 30mMria: an eventually consistent MnesiaDmitrii Fedoseev, Serhii Tupchii, Thales Macedo Garitezi, Zaiming Shi · Erlang 2023
• 40mKhepri - Mnesia replacementMichael Klishin · RabbitMQ Summit 2021
• 37mBreaking up with MnesiaDániel Szoboszlay, Onno Vos · Code BEAM V Europe 2021
• 41mConsistent, Distributed ElixirChris Keathley · ElixirDaze 2018
• 12mFault-Tolerant Replication with Pull-Based Consensus in MongoDBShuai Mu, Siyuan Zhou · NSDI ’21
• 15mKVIMR: Key-Value Store Aware Data Management Middleware for Interlaced Magnetic...Tsun-Yu Yang, Yuhong Liang, Ming-Chang Yang · USENIX ATC '21
• 38m(...) Phoenix on Riak_CoreBen Tyler · ElixirConf EU 2016
• 28mMongooseIM - The Right Tool for Scalable MessagingMichał Piotrowski · Erlang User Conference 2015