Efficient and reliable microservices

on YouTube ↗01h 14m

" summary

Claims embedding Cassandra in the JVM and collocating a custom in‑memory store removes most marshalling, memcache and network RTT costs and yields much lower CPU and latency for a messenger workload that must handle ~100k reads / 8k writes/sec and ~0.5T messages (~100 TB). Implementation details: embed Cassandra in JVM nodes act as data owners with RF=3, use hinting/read‑repair/streaming, expose mutation listener callbacks to refresh in‑memory caches, use shared memory mmap for fast restart, hook Cassandra compaction to build Lucene indexes, and likes/messenger clusters run at hundreds of instances and work in production.

AI-written summary · about

" chapters (34)

1. 0:00Intro
2. 0:48Messenger as an example
3. 1:39Chat Messages: storage
4. 2:00Chat Messages: full text search
5. 2:12Chat Messages storage service
6. 3:25Micro-service architecture
7. 6:13Micro-services: costs
8. 9:05Micro-services: lowering costs
9. 10:10Stateful micro-services
10. 11:52What can will go wrong?
11. 13:59Failures
12. 14:42Failure scenarios
13. 15:55Downtime probabilities
14. 18:38Embedding the Database
15. 21:05Request routing
16. 22:53Data partitioning
17. 25:29Messenger: calling the DB
18. 28:21Messages in-memory store:getMessages
19. 29:17In-memory store: freshness
20. 34:14Messages in-memory store: state loss
21. 36:38In-memory store: optimizing normal restarts
22. 37:47In-memory store: waiting for consistency
23. 39:33In-memory store: summary
24. 40:53getLastMessages( chats[])
25. 42:25split & merge
26. 43:57getLastMessages: local reads
27. 45:24search, indexMessages: Full Text Search
28. 47:50Compaction
29. 49:23Longer deployments
30. 51:13More DB nodes restarts
31. 52:01Longer scaleout time
32. 53:30Imbalanced resource utilization
33. 55:46Diagnostics and support
34. 57:34Stateful Services

" related talks

• 59mEfficient reliable microservicesOleg Anastasev · SnowOne 2020
• 40mHow to Build Ultra-fast Java Microservices. Overcome Pitfalls of Data PersistenceOtavio Santana · JCON WORLD ONLINE 2023
• 53mLeveraging Caching For Cloud Native ApplicationsMark Richards · Developer Summit 2020
• 54mRethinking REST in a Microservices WorldJames Roper · Philly ETE 2016
• 45mMigrating Mission Critical Workloads from Mainframe to Open...Venkat Mopuri, Seshu K Guddanti · Critical Software Summit @ Open Source Summit North America 2023
• 41mBuilding blocks & how to use them: MongooseIM case studyNelson Vides · Code BEAM Lite A Coruña 2022
• 35mCaching at Netflix: The Hidden MicroserviceScott Mansfield · Strange Loop 2016
• 47mArchitectural Patterns for Caching MicroservicesRafal Leszko · Developer Summit 2020