Recursion Schemes Without the Barbed Wire (Part 4)
summary
Recursion schemes power a concrete data-ingestion pipeline: validate semi-structured JSON/CSV schemas, transform to Parquet or Avro, and compose schema flavors without hand-writing converters. Algebras and co-algebras are implemented and composed so that conversions are round-trip property testable with property-based generation of arbitrary schemas; composition yields no whole-tree materialization and thus efficient streaming transforms. The workshop shows generators for schemas, a concrete invariant (schema -> Avro -> schema), and acknowledges no use for zygohistomorphic-prepromorphism.
AI-written summary · about
chapters (4)
- 0:00Schema transformations
- 5:26Property-based testing
- 10:51Generating arbitrary schemas
- 19:12Advanced topics
related talks
29mSchemas: The Key To Data Happiness
36mData pipelines engineering made simple with Scala
30mContracts, Schema Evolution in Data Pipelines
24mLightning Talks
21mPractical compositional diagramming
38mAdd ALL the things: abstract algebra meets analytics
38mLightning Talks
50mData Contracts VS Data Chaos: A Survival Story