Applying Functional Programming Patterns
summary
Demonstrates using Monoids to do a one-pass monoid fold over an RDD in Apache Spark that returns a tuple (example: 7 words, 28 chars and a word->count map) and shows swapping the tuple to add max/min or other metrics with no code rewalk; reports the monoid version has more allocation but is almost as fast as handwritten; shows replacing try/catch with Either/Validated/Ior so validation either accumulates all errors with Validated (NonEmptyList) or emits warnings with Ior; notes Validated is applicative not a monad.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:01Lego vs Duplo
- 4:05Monoid Laws
- 4:38Monoid Typeclass trait Monoid (A) def empty: A
- 5:31Using Our Monoid
- 6:58Monoid Zoo
- 9:08Monoids Compose (Lego Principle)
- 11:37RDDs and Folds
- 12:51The Program
- 13:48Running this program
- 15:19Easy Extension
- 16:23The new program
- 16:56More Monoid Tricks
- 18:12The Traditional Way
- 19:24The Functional Way
- 19:46Out Of The Box
- 21:54Example Time
- 22:19Validating Passwords
- 24:40Passwords with Validated
- 25:34Using Validated
- 27:16First Class Errors
- 28:07Enough Duplo, time for Lego!
related talks
48mMonads and Monoids: from daily java to Big Data analytics
45mRailway Oriented Programming - A Functional Approach to Error Handling
26mEssential Functional Structures: Functor, Applicative and Traverse
43mA Typeclass Primer for OO People
21mFP for Data Science For Loops Considered Harmful
29mTreasure.map(): Functional Programming in Kotlin
48m10 tips to write fast stack-safe functional code in Scala: the automatic-batch…
47mWhirlwind Tour of the Scala Functional Programming Ecosystem