AI-assisted coding, Python and dbt: taking Chronos live in two days
summary
A project produces 25 years of gap-free half-hourly weather data in two days by combining Python, dbt and Chronos-2 with Databricks, targeted prompting and short refinement loops rather than blind zero-shot runs. Intraday gaps use linear interpolation for intraday gaps and longer holes use Chronos-2 for multi-day gaps with the intraday-filled history fed back into Chronos. The result is a reproducible dbt pipeline that mostly succeeds, leaving a few edge cases to handle manually.
AI-written summary · about
chapters (12)
- 0:00AI-assisted coding intro
- 1:16Project overview
- 3:01Why gap-free weather data
- 6:12Traditional gap-filling methods
- 8:58Chronos zero-shot forecasting
- 13:53AI pipeline with Python and dbt
- 17:06Lessons from AI-assisted dev
- 21:25Key takeaways and future
- 23:12Q&A on data resolution
- 27:03AI agent access and production
- 29:11Handling weather anomalies
- 32:02Closing remarks
related talks
26mTime Series Analysis in Python: Easy Tools for Scientific Insight
30msktime: python toolbox for time series- next-generation AI
28mWorking with dates and times in Pandas
32mEnabling real-time insights through stream processing in Python
37mAccelerating Python Analytics by In-Database Processing
28mUsing Python in a Data Hackathon
30m5 Steps to Speed Up Your Data-Analysis on a Single Core
42mMaking Sense Out of Flight Test Data with Python