A Data Harmonization Engine on top of Django - The Good, the Bad, and the Ugly
summary
An open‑source data harmonization engine called data Hub runs on Django and Python, built by a two‑person team with one developer, and harmonizes raster, vector, CSV/Excel and API inputs into administrative units for mapping. Processing routines live as user‑authored Python files matched to a model key and are executed dynamically; Django admin manages metadata. Exports use pandas exports to stream JSON/CSV/XLSX, maps use Leaflet with D3 color scales and Plotly.js for time trends, deployed with Docker and environment config via django‑environ.
AI-written summary · about
chapters (9)
- 0:00Introduction and speaker background
- 2:10Project overview and data harmonization needs
- 5:28Why Django for the project
- 8:56Django project structure challenges
- 11:02Data harmonization engine implementation
- 13:31Exporting data to researchers
- 15:38Frontend tooling and JavaScript dependencies
- 17:05Why Django was the right choice
- 19:32Q&A session
related talks
31mDjango for life (sciences)
31mDigitising 100 years of parliamentary data - Django as a framework for government data management
26mBuilding Dynamic Dashboards With Django and D3
54mDjangoCon 2008 Panel: Django in Journalism
38mA Global Team Tackles World Problems with Django 🧭
54mAdvanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
48mEvolving Django: What We Learned by Integrating MongoDB
53mDjango: a Data Shovel With a Future