Multi-tenancy for all workloads
summary
Every OpenSearch cluster is effectively multi-tenant and the recommended approach is a pragmatic siloed pooled hybrid mix: pool small tenants into single indices, split large tenants into their own clusters, and use a tenant routing proxy plus a database mapping with a read-only flag for reindexing to move tenants without downtime. Capacity math is concrete: 200×64 vCPUs = 12,800 vCPUs → theoretical ~1.28M QPS at 10ms, real-world peaks seen: 28k QPS and 1.3M metrics/sec on ~80-node clusters. Keep shards ~10–30GB (search), 50GB (logs), shard:CPU ≈1–1.5, free storage <15%.
AI-written summary · about
chapters (10)
- 0:00Multi-tenancy challenges
- 1:07Multi-tenancy in OpenSearch
- 2:34Resource allocation and workload design
- 14:18Siloed tenant strategy
- 21:38Pooled tenant strategy trade-offs
- 29:19Hybrid tenant strategy scalability
- 33:17Cell-based and hub-and-spoke architectures
- 36:50Scaling search workloads
- 37:53Traffic management for tenants
- 40:22Best practices for OpenSearch
related talks
44mBringing Down the Cost of Your OpenSearch Clusters
44mScaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...
24mReimagining Search : Scaling OpenSearch Across an Enterprise - Platform C...
31mMaximize Resource Efficiency with Separated Index and Search Workloads
28mRunning OpenSearch at Scale in High-Traffic Gaming Systems
31mSharding 2.0: Unlocking Infinite Search Scalability in OpenSearch
22mSharding 2.0: Unlocking Infinite Search Scalability in OpenSearch
45mMonitoring Your OpenSearch Cluster Effectively