Interview

on YouTube ↗03m

" summary

Builds and contributes to Apache Spark at Google and works across Airflow, SystemML, and Mahout, focusing on practical data pipelines that run in cloud environments. Argues that many teams skip adequate testing in cloud because service mocks are hard to create, so pushes concrete strategies to validate pipelines before production. Credits Scala and functional programming for enabling reliable pipeline code and describes real-world tradeoffs when shipping big data systems.

AI-written summary · about

" related talks

• 04mLong Cao InterviewAlexy Khrabrov, Long Cao · Scale By the Bay 2017
• 54mData engineering & DataOps: Ask Me AnythingLars Albertsson, Ellen Friedman · Berlin Buzzwords 2021
• 01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live codingOleg Klimenko · Java ON 2022
• 31mSpark challengesVitaly Polonetsky · Data Con LA 2021
• 03mFred Reiss InterviewAlexy Khrabrov, Frederick Reiss · Scale By the Bay 2017
• 07mThomas Gerber InterviewAlexy Khrabrov, Thomas Gerber · Scale By the Bay 2017
• 20mShift-left testing : Building reliable Data PipelinesHarmeet Sokhi · DataEngBytes 2021
• 41mSpeeding Up Data Pipeline Construction in Apache SparkFabiane Nardon · QCon São Paulo 2019