HTAP Workloads: Challenges and Solutions

on YouTube ↗01h 01m

" summary

Compares concrete solutions and tradeoffs: persistent-parallel trees give lockfree reads by versioning the whole index but CAS-based commits serialize writers unless parallel batching or combining is used to restore throughput, fork+COW (HyPer) achieves fast snapshots via process fork and copy on write, MVCC creates per-row versions with start/commit timestamps providing snapshot isolation while allowing writers to block long-running readers, and HTAP-aware LSM (LASER) places rows in top levels for OLTP and column or column-family layouts in lower levels for OLAP, with log shipping or WAL replication used to keep OLAP replicas current.

AI-written summary · about

" chapters (32)

1. 0:00HTAP workloads intro
2. 1:25Database workload types
3. 4:54Concurrency challenges in HTAP
4. 8:48OLTP vs OLAP conflicts
5. 9:52Lock-free reads and consistency
6. 12:02Persistent data structures
7. 14:48Universal construction limits
8. 16:35Parallel batching technique
9. 19:43Latency vs throughput trade-off
10. 21:09Persistent parallel trees reading list
11. 21:30Multiversion concurrency control basics
12. 23:15MVCC read handling
13. 24:40MVCC transaction commit
14. 26:05MVCC writer blocks reader
15. 27:30MVCC past read timestamps
16. 28:53MVCC garbage collection
17. 29:58Snapshot isolation guarantees
18. 30:41MVCC in modern DBs
19. 31:23Row vs column storage
20. 33:35OLAP with row storage
21. 34:41OLAP with column storage
22. 37:11Column families for mixed workloads
23. 38:34Separate OLTP/OLAP storage
24. 40:20Leader-follower replication
25. 41:23Log shipping and CDC
26. 43:11LSM-trees explained
27. 47:27LSM-trees in modern DBs
28. 48:52Merging blocks in LSM
29. 51:01Time-based LSM partitioning
30. 53:50Storage layout per level
31. 56:21HTAP storage solutions reading list
32. 57:25Q&A

" related talks

• 29mHTAP vs OLTP/OLAPPeter Zaitsev, Robert Hodges, Alexander Zaitsev, Ed Huang · OSA Con 2022
• 01h 13mHTAP made live — from the engineering perspectiveEd Huang · Hydra 2020
• 49mHTAP with Azure Cosmos DB: Hybrid Transaction & Analytical ProcessingHari Sudan S · Vaccination Database Tech Talks (Booster) - Spring 2022
• 14mRetrofitting High Availability Mechanism to Tame Hybrid Transaction/Analytical ProcessingRong Chen, Haibo Chen, Binyu Zang, Sijie Shen · 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21)
• 16mHolistic and Automated Task Scheduling for Distributed LSM-tree-based StoragePatrick P. C. Lee, Yongkun Li, Siyuan Sheng, Yuanming Ren, Zhang Cao · FAST '26
• 18mPreview: KV-Stores and DatabasesAlex Conway · OSDI '21 / USENIX ATC '21
• 36mHow Can HTAP Databases Simplify Your Data Infrastructure?Liquan Pei · OSA Con 2022
• 41mHTAP with MariaDB PlatformTodd Stoffel · Data Con LA 2019