Revolutionizing Data Engineering for Space Exploration
summary
Claims concrete workloads and numbers: hundreds of gigabytes per mission per day, ISS nearly a terabyte daily, James Webb about 40 GB/day, and systems handling 150 simultaneous requests. Describes distributed architectures, layered pipelines, and advanced compression with compression ratio as high as 10. Mentions edge processing with nodes drawing under five W per node, realtime actions at sub-10 ms latency, and proposals for orbital cloud, quantum-accelerated planning, and delay-tolerant interplanetary networking—what was built and the measured limits matter.
AI-written summary · about
chapters (12)
- 0:00Introduction to Space Data Engineering
- 0:37The Scale of Space Data
- 1:31Distributed Data Systems
- 2:20Advanced Compression Techniques
- 2:57Quantum Computing in Space
- 3:51Edge Processing for Spacecraft
- 4:44High Performance Computing Architecture
- 5:10Real-Time Data Processing
- 5:46James Webb Space Telescope
- 6:11The Economic Impact of Space Data
- 6:37Future of Space Data Systems
- 7:09Conclusion
related talks
55mAbove the Cloud: Building Data Centers in Space
45mDesigning Dope Distributed Systems for Outer Space with High-Fidelity Simulation
26mReimagine Real-time
30mRetooling for a Smaller Data Era
39mUnlock the potential of your data: Optimize your data store architecture...
36mThe MODERN Modern Data Stack: Building an Open Distributed Data Warehouse Beyond...
43mProcessing Data From The James Webb Space Telescope
33mManaging IoT Storage with Google's Cloud Platform