Building a Parallel, Distributed Tool for Multi-Terabyte Backup
summary
Describes building a parallel, distributed backup tool for multi-terabyte datasets using Python; covers technologies and design decisions used to scale backups across multiple machines and disks. Explains strategies for parallelism, data distribution, fault tolerance and performance optimization specific to large-scale backup workloads.
AI-written summary · about
related talks
38mScale out from the very beginning
40mScaling Python - Bank Edition
30mThe BorgBackup Project
01h 01mParallel Python Analyzing Large Data Sets
01h 03mDask: Next Steps in Parallel Python
08mPyTorch Distributed: Towards Large Scale Training
35mData Warehousing with Python
25mDeploying Python at Scale with Dask