Beyond the ORM: from Postgres to OpenSearch with Andrew Mshar
summary
Iris moves address data out of Postgres into Cosmos on OpenSearch with Greenhouse as the API layer and equipment followup in progress, cutting a worst case SQL query from about two minutes to a few hundred milliseconds and making large imports far faster, all on AWS OpenSearch using opensearch-py and Cabana dashboards. Development adopts Pydantic models replace the ORM for validation and typing, trading Django admin for locations moved fully to OpenSearch, gaining query dropped from ~2 minutes to ~200ms, and using opensearch-py and Cabana dashboards for iteration.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:06Business context
- 6:26Iris app challenges
- 11:45Moving data to OpenSearch
- 13:10Why OpenSearch over PostgreSQL
- 15:42OpenSearch setup
- 17:05OpenSearch data operations
- 21:42Performance issues with queries
- 23:31OpenSearch query structure
- 25:17Pre-migration results
- 27:03Trade-offs beyond ORM
- 31:18Summary and Q&A
related talks
39mOpenSearch Helps You to Adopt a Friend
28mSuper Search with OpenSearch and Python
38mMigrating from Self-Managed ElasticSearch to OpenSearch
09mIntroducing Opensearch in existing multisource enterprise App
44mScaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...
31mMigrating from Self managed Elasticsearch to OpenSearch & How Indias Fantasy Sports Giant Leveled Up
35mInnovating at Scale: Lessons from Pegasystems' OpenSearch Journey
22mImproving the efficiency of more than 800 databases with o11y