Interview

on YouTube ↗03m

" summary

Spotify uses Scio as the default Scala-based data processing framework for batch and streaming pipelines, built on Apache Beam and adopted across teams. The talk claims Scio yields more robust and sometimes faster jobs by leveraging the type system and functional patterns, and describes practical lessons from eight years of Scala use, migrations of code, and interactions with libraries like cats. Demos show production pipelines running and concrete API choices that reduced bugs and improved performance.

AI-written summary · about

" related talks

• 38mScaling ScalaJulien Tournay, Filipe Regadas · Scala Love in the City 2021
• 36mData processing at Spotify - Why portability mattersJulien Tournay, Filipe Regadas · Flink Forward Global Virtual 2020
• 04mNeville Li InterviewAlexy Khrabrov, Neville Li · Scale By the Bay 2017
• 54mData engineering & DataOps: Ask Me AnythingLars Albertsson, Ellen Friedman · Berlin Buzzwords 2021
• 04minterviewUmayah Abdennabi · Scale By The Bay 2019
• 34mFrom Data Centers to Cloud Native - Dave Zolotusky & James Wen, SpotifyDave Zolotusky, James Wen · KubeCon + CloudNativeCon Europe 2018
• 10mDemocratizing the pipeline with Apache Beam and DataflowTyler Akidau, Reuven Lax · Strata Data Conference 2017 - New York, New York
• 03mChris Phelps InterviewAlexy Khrabrov, Chris Phelps · Scale By the Bay 2017