Package Management for Data Scientists
summary
Compares concrete failure modes and fixes for R and Python: version conflicts illustrated by Rcpp 1.0.6 vs 1.0.7, and missing system libraries like OpenSSL/GCC breaking pip installs. Recommends reproducible artifacts such as renv lock and requirements.txt, snapshot mirrors like Posit Public Package Manager, and environment tools like Docker, Conda or Pipenv. Advises preferring wheels/prebuilt binaries, pinning versions, producing lockfiles for every script, and using CI tools (eg Dependabot) to safely exercise upgrades.
AI-written summary · about
chapters (6)
- 0:00Introduction and scope
- 1:42Package management basics
- 2:39Conflicting version challenge
- 6:17System dependency challenges
- 8:30Best practices and takeaways
- 9:41Q&A session
related talks
48mPicking a Winner: How to Pick the Right Dependency (Resolution...
58mHow Can Package Managers Handle ABI (In)compatibility in C++?
05mData Quality and Validation in ML Pipelines
35mData versioning in machine learning projects
20mAutomating for Consistency
01h 08mUsing and contributing to the data.table package for efficient big data analysis
26mReproducible Postgres
42mRobust Automated Forecasting in Python and R