Visualising Public Service: Seeing Inside Birmingham City
summary
Birmingham Civic Dashboard visualizes call centre dumps, ingesting ~10,000 interactions a day from SAP, importing into SQL, aggregating into daily MongoDB documents and serving a Rails front end on Heroku. It anonymizes by using truncated postcodes and snapshots tiny JSON fragments for performance, solves deep nested-schema quirks with bespoke domain logic, and overcomes the Heroku 30-second limit by streaming CSVs. The site works but required editorial mapping and ongoing schema fixes.
AI-written summary · about
chapters (15)
- 0:00Introduction to project
- 1:16Birmingham Civic Dashboard demo
- 3:09Data pipeline overview
- 7:28Organizational structure challenges
- 10:43Privacy and personal data issues
- 13:11Data exploration and normalization
- 14:56Nested data structures in Ruby
- 17:47Performance optimization with snapshots
- 19:15Editorial and display limitations
- 21:48Prototyping and deployment lessons
- 24:54Map visualization and KML issues
- 26:19Heroku platform insights
- 28:25Data reliability assumptions
- 30:29Project outcomes and reception
- 30:53Q&A session
related talks
22mGovernment dashboards and open data.
26mBuilding Dynamic Dashboards With Django and D3
40mOpen Data Dashboards & Python Web Scraping
18mData and the City
21mTracking COVID 19 with Ruby
43mBehind the Scenes at Gapminder: A Story about Data and Visualizations
35mDesigning for Impact in Civic Data Science
40mA Few Good Public Servants: How Great Analysis Inspires Action