Keeping Data Alive: Persistent Storage Options for Dynamic Cloud Applications
summary
A small Shiny/Streamlit aquarium app is extended to record user ratings and reaches v1.1 by wiring persistent storage: Google Sheets for quick free setup and real‑time updates but row/column and rate limits; DuckDB on S3 for fast reads (concurrent reads only, writes lock); MotherDuck for DuckDB-compatible hosted storage that queues concurrent writes; and a Postgres managed DB for robust multiwrite durability at roughly $6–$15/month. Posit Connect Cloud stores encrypted secrets to connect these services.
AI-written summary · about
chapters (8)
- 0:00Persistent storage intro
- 1:43Why persistence matters
- 3:29Static vs dynamic data needs
- 6:20Storage options for dynamic data
- 11:21Managed Postgres choice
- 13:24Persistence in demo app
- 15:51Data science vs web app needs
- 16:34Q&A: Migrations & DuckDB writes
related talks
36mMotherDuck
27mMotherDuck: Taking flight with interactive analytics
38mFrom Big Data to DuckDB: Ryan Boyd (MotherDuck) on Hyper-Tenancy, Duck Lake, and Making Data Fun.
51mMotherduck: Data Analytics in the Post Big-Data Era
24mData infrastructure to build bigger with less
23mWhat's an App Without Data?
05mDuckDB Powering Interactive Notebooks
55mDuckLake: Learning from Cloud Data Warehouses to Build a Robust “Lakehouse”