On 11/1/23 20:01, rajani m wrote:
The search query "http://host:8983/solr/v9/select?&q=*&rows=10"; consistently took ~500 ms. With "distrib=false" all the 96 shards have QTime 0-25 ms. Does this mean aggregation of results from all the shards is taking ~475 ms? I also tried shards.rows=5 and it still returned in ~475 ms query time. I am assuming the sort for a star query is by score, is that adding to such high latency? Why would aggregation take so long? When I do "debug=true"
A query of "*" is a wildcard query. These queries are naturally slow. That query gets expanded to "default_field:*".
If you are wanting to query for all docs, use the special syntax of "*:*" (without the quotes). Despite the presence of the asterisks, this is NOT a wildcard query, and it runs VERY quickly.
A wildcard query is slow because of how it actually functions behind the scenes. First the system must query the index to find all possible terms on the field. Then it must execute a query with all of those possible values. With a query of "*", if the field has a billion terms, then the internal query object will contain a billion values. It is quite slow.
On my tiny Solr install with emails indexed from dovecot (currently at 306811 docs), a query of "*" (df is set to the body field) had a QTime of 31 milliseconds. A query of "*:*" only took 11 milliseconds.
Thanks, Shawn
