Mining Large Data Sets of Biomedical Omics Da...
summary
EMBL processes spatial metabolomics images with >10,000 channels and experiments often >1 TB, and Metaspace now handles >10,000 submissions with recent superlinear growth. They replace Spark clusters with IBM Cloud Code Engine plus the Lithops serverless framework, turning embarrassingly parallel analysis into on‑demand batch jobs that scale to zero, remove queue and deployment delays, and cut infrastructure planning. The pipeline runs at scale, preserves analytic fidelity, and demonstrably lowers operational overhead and time-to-result for users.
AI-written summary · about
chapters (5)
- 0:00Start
- 4:10Metaspace
- 6:20Deployment Delay
- 6:32Infrastructure Management and Queue Management
- 13:13What Code Engine Does for You
related talks
25mScaling Up Genomics with Spark
45mDigital Disruption in health & medical research
50mCancer Genomics - a biologist and a developer
42mHow Novel Compute Technology Transforms Life Science Research
19mBig Data
08mRevolutionizing Healthcare with Big Data and ML
36mAn almost perfect match: cloud native and scientific computing
29mUsing Big Data And Ai In Transport And Logistics