Using and contributing to the data.table package for efficient big data analysis
summary
An R package with C core claims dramatic practical speedups: fread is fastest on large CSVs, summarization benchmarks show up to 100× faster than base R and roughly 2–3× faster than pandas/polars, and CSV writes can be ~10× quicker. Key tools demonstrated are setkey joins, rolling and non‑equi joins, melt/dcast reshaping and in‑place := updates that avoid copies. The project supports CI performance checks, nightly reverse‑dependency testing, and community contributions including translation and travel awards.
AI-written summary · about
chapters (9)
- 0:00data.table intro
- 7:12Why data.table is powerful
- 15:45Install and use basics
- 27:55Advanced I J operations
- 34:21Joins in data.table
- 44:22Reshaping data
- 57:04data.table in packages
- 1:00:16Contributing to data.table
- 1:06:40Summary & Q&A
related talks
20mGive me the Data
18mLooking under the hood - demystifying data tools
58mData Wrangling [for Python or R] Like a Boss With DuckDB
19mDatabases using R The latest
44mKeynote: Working Efficiently with Big Data in Text Formats
01h 25mIntroduction to Data Manipulation and Visualizations in R
31mData wrangling like a boss with Duckdb
01h 31mPandas, Data Wrangling & Data Science