Transducing Healthcare: Transducers as Clinical Features
summary
Sojourner is a Clojure/Datomic streaming feature engine that ingests HL7 from 180+ hospitals and produces reusable clinical features for about 40 million patient encounters per year. Architecture uses a lambda of Clojure microservices on Kafka and GKE, aggregates noisy HL7 into indexed patient collections, and compiles declarative configs into Clojure transducers that emit features to BigQuery and topic consumers. It processes ~30–40 TB/week and ~250k messages, and powers SOFA scoring, discharge dashboards, and LLM handoffs.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 0:55Hospital Context
- 3:32Data
- 11:26Workflow
- 15:50Sojourner realtime pipeline
- 29:05Feature Delivery
related talks
36mSynthea: Massive FHIR Data
33mHealthcare "big" Data from Data Management to Web Apps
33mClojure Dispatches From a National Research Lab
54mUntangling Healthcare with Spark and Dataflow
41mAccelerating privacy-enhancing data processing
45mHow to solve scaling pains with FHIR and data streaming?
36mSimplifying ETL with Clojure and Datomic
21mClinical and Claims Data Transformation utilizing FHIR Bulk Data