CETOFS: A High-Performance File System with Host-Server Collaboration for Remote Storage

on YouTube ↗14m

" summary

CETOFS implements a userspace-kernel collaborative architecture for NVMe-over-RDMA remote SSDs, moving the data path into user space and offloads permission checking, concurrency control, failure-atomic IO to the storage server. It runs on RDMA and Optane-class SSDs, implements block-granularity owner maps, request-group creation and merging, and transaction-aware batching. Compared to kernel Ext4 and a userspace FS it cuts single-thread read/write latency by up to 52% and raises concurrent throughput up to 19×, demonstrating the prototype works and scales.

AI-written summary · about

" chapters (13)

1. 0:00CETOFS introduction and motivation
2. 1:00Remote SSD access workflow
3. 2:44Software overhead in remote storage
4. 3:48Scalability with concurrent access
5. 4:51Atomic IO over remote storage
6. 5:53CETOFS architecture design
7. 6:55Permission checking in CETOFS
8. 8:20Concurrency control with grouping
9. 10:52Merging request groups
10. 11:34Atomic IO and transactions
11. 11:56Evaluation setup and metrics
12. 12:40Results: Kernel overhead and scalability
13. 13:02Conclusion and future EFS work

" related talks

• 17mFisc: A Large-scale Cloud-native-oriented File SystemChao Han, Xiaoliang Wang, Qiang Li, Qiao Xiang, Jiesheng Wu, Jiwu Shu, Jiaji Zhu, Yuanyuan Dong, Zhaosheng Zhu, Yaohui Wu, Shaozong Liu, Zhongjie Wu, Junping Wu, Zheng Cao, Jinbo Wu, Lulu Chen, Shuo Huang, Wenhui Yao, Minfei Huang, Puyuan Yang, Shanyang Liu, Huayong Wang, Haonan Qiu, Derui Liu, Yujie Zhou, Zhiwu Wu, Shang Gao, Zicheng Luo, Yuchao Shao, Gexiao Tian, Jie Wu · FAST '23: 21st USENIX Conference on File and Storage Technologies
• 34mNVMeoF in Ceph - What's New and Performance Best PracticesAviv Caro, Mike Burkhart · Cephalocon 2025
• 14mFlexTOE: Flexible TCP Offload with Fine-Grained ParallelismSimon Peter, Rajath Shashidhara, Timothy Stamler, Antoine Kaufmann · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 30mOrion: A Distributed File System for Non-Volatile Main Memory and RDMA-Capable NetworksJian Yang · FAST '19
• 39mSecured I/O in the Cloud with NVMe-Over-TCPHannes Reinecke · LinuxCon @ OSS NA 2023
• 16mNo Provisioned Concurrency: Fast RDMA-codesigned Remote Fork for Serverless ComputingRong Chen, Haibo Chen, Xingda Wei, Jinyu Gu, Fangming Lu, Tianxia Wang, Yuhan Yang · OSDI 2023
• 49mPetabyte CephFS for Satellite Data ProcessingMartin Strigl · DevConf.CZ 2020
• 17mCo-Designing Traffic Control with NVMe-oF for Disaggregated Storage: A Comparative StudyMing Liu, Chendong Wang, Joontaek Oh · NSDI '26