Scaling deliberation with data science and Clojure
summary
Built a working deliberation system called Polis that models opinions by applying dimensionality reduction (PCA-style) and k‑means clustering to votes on many comments, producing visual opinion spaces and surfacing unexpected consensus (example: a 119-person cluster identified distinguishing statements). Deployed at national scale in vTaiwan to inform legislation (helped shape rules for Uber and Airbnb), rewritten in Clojure for concurrency and JVM interop, open‑sourced, now a nonprofit but facing scaling, ops and maintenance challenges.
AI-written summary · about
chapters (18)
- 0:00Start
- 1:29Scaling deliberation with data science
- 2:56Existing tech
- 3:42Opinion space • Dimension reduction (PCA) • Clustering K-means; Lloyd's
- 9:15Polis Technologies Inc
- 10:04Enter Clojure
- 13:002014 - Sunflower Movement
- 15:28A platform for rational discussion and deliberation
- 15:59v Taiwan
- 17:17Press
- 18:21Open source
- 18:39Challenges
- 20:03Back to Clojure...
- 20:15The good
- 20:49The awesome (potential) • ClojureScript Vega-Lite & Vega adoption for visualisation
- 23:17The ugly
- 25:20Difficult questions
- 26:53Starting volunteer drive
related talks
42mClojure on the cyberpunk frontier of democracy
43mDoing Data Science with Clojure
41mHelping voters with Pedestal, Datomic, Om and core.async
26mDoing data science with Clojure: the ugly, the sad, the joyful
24mData science, apps and civic responsibility
24mThe Return of Clojure Data Science
33mBuilding Conversational Speech Annotation Tool in Clojure
44mData Science in Clojure