Productionize your open-source project
summary
Builds an automated pipeline that refreshes COVID‑19 data nightly from Johns Hopkins, runs unit checks, packages code in Docker, and deploys a live tracker to Github Pages using scheduled Github Actions workflows (cron every ~8 hours). Shows the surprising scale-up from a one‑person script to daily automation, what tests catch (schema, ranges, negative values), how images are pushed to a registry, and limits (tiny runners, community support, potential paywall).
AI-written summary · about
chapters (16)
- 0:00Productionizing open source
- 2:12Why productionize open source
- 8:57Manual data pipeline problems
- 11:27Automated API and pipeline
- 13:57GitHub Actions and Docker
- 16:26Docker reproducibility benefits
- 17:09Docker local build and test
- 18:13GitHub Actions testing workflow
- 19:17GitHub Actions data updates
- 20:42Unit testing for data
- 21:48GitHub Actions limitations
- 23:16Long-term maintenance
- 24:19Docker for data science
- 25:23Debugging test results
- 25:44Knowledge sharing in open source
- 27:12Q&A
related talks
45mLet's automate one more thing
25m20 Minutes to Build a Serverless COVID-19 GraphQL API
28mDevOps Stories; Think outside of your Pipelines
47mUnder the Radar: How we found 0-days in the Build Pipeline of OSS Packages
29mOur Journey to Fully Automated Weekly Releases
45mWorkshop: Deploy and Monitor ML Pipelines with Open Source and Free Applications
01h 11mDeploy & Monitor ML Pipelines with Python, Docker and GitHub Actions
59mDevOps for the Discouraged