Kafka and the REPL: Stream Processing, the Functional Way

on YouTube ↗35m

" summary

Funding Circle describes a Clojure wrapper for Kafka Streams called Jackdaw that they release and use in production to process loans and payments (platform has originated £5.6B and moves millions of dollars daily). Demos include a word‑count (Metamorphosis input) and a double‑entry ledger that produces transactions, attaches type 5 UUIDs, materializes balances with KTables backed by RocksDB, and runs end‑to‑end tests; shows live restarts, partitioning, deduplication and schema/versioning gotchas.

AI-written summary · about

" chapters (8)

1. 0:00Introduction to Funding Circle
2. 0:33Kafka concepts and basics
3. 2:53Kafka streams and topology
4. 7:01Why Clojure for stream processing
5. 8:38Introducing Jackdaw
6. 10:50Demonstrating stream processing
7. 19:28Advanced ledger examples
8. 29:45Kafka development gotchas

" related talks

• 42mBuilding a Data Pipeline with Clojure and KafkaDavid Pick · Clojure/conj 2014
• 33mClojure Dispatches From a National Research LabJennifer Lewis · Clojure/west 2017
• 46mClojure Scaling the Event StreamDerek Troy West · Clojure/conj 2017
• 31mOne Million Clicks per Minute with Kafka and ClojureDevon Peticolas · Clojure/west 2016
• 37mCascalog 2- Streaming MapReduce in ClojureSam Ritchie · Clojure/conj 2013
• 43mDoing Data Science with ClojureSimon Belak · Lambda World 2016
• 26mDistributed Computing in Clojure and SparkJonathan Brown · Clojure/north 2019
• 37mDon't Shed a Tier for Me, ClojureDavid Pollak · Clojure/conj 2013