Parallel SQL & Streaming Expressions in Apache Solr 6
summary
Solr 6 exposes Parallel SQL compiled to a Streaming Expressions plan and served over /sql and /stream with a thin JDBC client; queries run in two modes — map-reduce (hash-partitioned, /export-driven shuffling that requires docValues and uses worker collections) or facet-driven aggregation. Joins can span collections and external JDBC sources, all replicas participate, and the exporter reports about 800,000 tuples/sec per node for shuffles. Limitations: no nested-object support, no built-in pagination, and the feature is new and still rough.
AI-written summary · about
chapters (7)
- 0:00Introduction to Solr
- 1:31Parallel SQL overview
- 6:12Query execution strategies
- 14:03Implementation details
- 22:12Shuffling and performance
- 27:19Architecture and Q&A
- 34:00Future directions
related talks
35mBuilding analytics application with streaming expressions
50mAnalytics and Graph Traversal with Solr
21mStream Processing and SQL
19mVendor-Agnostic Analytics: Querying Open Columnar Formats With OpenSearch
41mStreaming SQL
42mUsing OpenSearch as a Database With DataFusion
33mStreaming Databases
36mSolrCloud and NoSQL