What we learned from Papermill to operationalize notebooks
summary
Claim: papermill compatibility plus new hooks make notebooks operational rather than just documents. Implementation details: papermill python API and CLI for injection, an Azure Data Studio URI parameterization handler that reads query strings, a run with parameters action that injects and executes parameter cells, KQL magic and plotly demos over ~59,000 storm events that authenticate and return results, and a prototype for global Jupyter Book parameters proving the flow end to end.
AI-written summary · about
chapters (7)
- 0:00Intro
- 0:47Internal troubleshooting
- 5:49Parameterization Demos
- 9:31Notebook URI Parameterization
- 11:49Run with Parameters Action
- 19:19Global Parameterization
- 24:42Questions?
related talks
39mData and ETL with Notebooks in Papermill
32mCombining IPython with Open Source Papermill to enhance your Jupyter Notebook
34mNotebooker: Production and Scheduling for your Jupyter Notebooks
28mCreating custom engines for flexible notebook execution
01h 23mImproving Your Data Modeling Work Through Open-Source Software
36mThe Lifecycle of a Jupyter Environment
01h 25mTroubleshooting your Data Workflows with Noteable+Dagster-live debugging of failed jobs
25mThe First Notebook War