Fred Reiss Interview
summary
Work on Apache Spark focuses on scaling transformations for very wide tables, for example recoding string columns when there are millions of distinct values, by pushing work into the built‑in SQL engine instead of shuffling massive data. Efforts include making ML run at cluster scale, integrating deep learning via add‑ons like TensorFlow on Spark and TensorFrames, and changing architecture to avoid inefficiencies when shipping functions to every node.
AI-written summary · about
related talks
46mSpark: One Tool for Everything?
55mSpark Architecture
52mSolving Analytical Problems using Apache Spark
45mFrom Big Data to Fast Data: Apache Spark
23mRethinking scalable machine learning with Spark ecosystem
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
16mPetabytes of data: How to use R at scale
50mIntroduction to Real-Time Big Data with Apache Spark