Evaluating fire simulators using Docker, Dask and Jupyter notebooks!
summary
Built a reproducible evaluation pipeline that wraps heterogeneous fire simulators (Prometheus, Phoenix, SPARK, Australis) in Python, task‑farms hundreds of runs across a mixed Windows/Linux cluster, and compares outputs to line‑scan polygons for objective metrics and sensitivity tests (e.g. wind direction sweeps). Orchestration uses Docker containers and docker‑compose, execution uses Dask distributed with delayed graphs and a progress UI, and analysis + reports run in Jupyter notebooks. Results are repeatable via GitHub, Docker Hub and CI (CircleCI) and the workflow successfully regenerates published experiments.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:30Presentation
- 3:00Why are simulators important
- 4:40Demo
- 9:00Fire tornado
- 12:20Evaluation
- 15:56What supports an evaluation
- 20:17Docker Environment
- 21:36Dask
- 22:25What is Docker
- 23:41Simulator execution
- 24:12Delayed decorator
- 24:44Executor
- 26:00Why open source
- 26:17Crossplatform workers
- 27:07Dask vs Spark
- 27:48Jupyter notebooks
- 28:44Visualising data in notebooks
- 29:42Reproducibility
related talks
20mBuild an e2e analytics application using DataHub
42mSimplifying Data Analysis with GitHub Codespaces, Jupyter Notebooks & Open AI
35mSimplification of Data Pipelines by Containerization
37mPatterns for Collaboration Between Data Scientists and Software Engineers
34mMoving notebooks into the cloud: challenges and lessons learned
37mPanel Discussion
43mDeploying Large Spark Models to production #bbuzz
46mThe Python Data Ecosystem: Navigating a fragmented landscape