Realizing the Full Potential of NOAA’s Open Data
summary
NOAA reports 30 petabytes in archive plus ~200 PB in working stores, and pilots moving datasets to partners under cooperative research and development agreements with Amazon, Microsoft, Google, IBM and Open Commons Consortium. Weather radar (NEXRAD) on AWS shows about 2x usage and a 50% drop in load on NOAA servers, 80% of radar users pulling from AWS, BigQuery hosts petabyte-scale climate tables, CloudFront handled over a billion hits/day for Hurricane Irma. Licensing via community data agreements is under consideration.
AI-written summary · about
chapters (27)
- 0:00Intro
- 0:13A science background
- 1:05NOAAs mission
- 1:55Big data
- 2:34Popular data sources
- 3:09Datagov
- 4:11Open Source
- 4:59World Weather Organization
- 5:48Tension between government and industry
- 7:35NOAAs partnership policy
- 8:07NOAAs licensing policy
- 9:06The value of NOAAs data
- 10:20The ocean economy
- 10:44The weather enterprise
- 11:28The experiment
- 12:11How much is going out
- 12:42Demand is growing
- 13:17Open Commons consortium
- 14:31Data broker
- 15:10Publicfacing services
- 15:29Weather radar data
- 16:46BigQuery
- 17:17OCC
- 17:28Tampa Bay Times
- 18:09IBM
- 18:43Fake Forecast
- 19:34Summary
related talks
17mIt's All in the Cloud
21mCloud-Native Data in Practice
22mScientific Data Under Threat in Today’s America
05mAlex Kudriashova Interview
19mThe Power of Open
21mAquawatch Data Systems a cloud solution for water quality monitoring and forecasting
10mPaco Nathan (Liber 118) Interview - OSCON 2014
41mSnakes in the Microwaves: How Python is Powering the Golden Age of SAR