There is a missing tool in your Java data structure toolkit – DataFrames!
summary
Compares Python pandas, TableSaw, Kotlin dataframes and DataFrame‑EC on the 1 billion row challenge and a 100 million test and shows pandas runs fast but uses huge memory while Kotlin fails at 1B, TableSaw loads faster but aggregates slowly, and DataFrame‑EC uses Eclipse Collections primitive lists and Eclipse Collections pooling to cut memory and often halve aggregation time versus TableSaw; code and reproduction live in the linked GitHub repo.
AI-written summary · about
chapters (15)
- 0:00Introduction to Data Frames
- 1:50Data-oriented programming
- 5:15OO vs. Data-oriented domain
- 9:53What is a data frame?
- 11:52The 1BRC challenge
- 15:50Data frames for developers
- 18:36Pandas solution analysis
- 21:29JVM data frame libraries
- 24:02Tablesaw and Kotlin frames
- 27:01Performance comparisons
- 31:01Dataframe-EC features
- 32:29Eclipse collections backend
- 38:59Donut shop case study
- 45:00Computed columns and pivot
- 47:17Conclusion and resources
related talks
54mDataframes, Collections and Streams in Java
31mData Science Yes, But in Kotlin!
41mKotlin for Data Science
32mJava's Ignored Potential: Supercharge Database Apps with Java In-Memory Process
47mJava's Hidden Gems: Tools and Libraries
35mWorking with data in Python for the Aspiring Data Scientists
38mData Analysis Yes, But in Kotlin!
45mJava is the “big” in BigData