Monitoring Displacement Crises with Python
summary
Python backend that scrapes web pages and runs an NLP pipeline that extracts numbers of people displaced and timestamps, using newspaper for parsing, gensim for topic reduction, scikit-learn for classification and sqlalchemy for storage. Prototype ingests heterogeneous news, separates conflict v. disaster, handles multiple events per article, and improves F1 with combined rules + ML. Work targets the Internal Displacement Monitoring Centre workflow and moves toward integration with their monitoring suite.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 3:06Data for Democracy and displacement crises
- 11:34Natural language processing for classification
- 15:08Extracting displacement event details
- 17:58Collaboration and outcomes
- 20:49Reflections on civic tech
- 23:19Q&A: Granularity and geo-tagging
- 24:04Q&A: Duplicate and inaccurate reports
- 25:50Q&A: Follow-up projects and language support
- 27:17Q&A: Scaling and funding
- 29:26Q&A: Interactive maps and visualizations
related talks
43mPython at City Scale
37mData Science for Digital Humanities: Extracting meaning from Images and Text
26mFrom Data to Decision: Monitoring and Improving Water Networks using AI/ML
22mPython Applications in Infrastructure Planning and Civil Engineering
38mKeynote: Using the Python Data Science Stack to Determine
25mCitizen Science: Tracking Aliens with Python!
34mVisualising NLP pipelines with Pynorama
18mAutomate Your City Data with Python