Do you really have 96 separate disks and memory for each shard? They seemed a bit small and numerous to me, unless you are trying to fit every shard into memory of separate nodes and have the hardware resources for it
— > On 19 Apr 2023, at 05:43, Rajani Maski <rajinima...@gmail.com> wrote: > > It is a query with popularity and recency boosts, requesting the first 100 > docs with 3 fields per doc. No facets. It is a query against a collection > of 96 shards ~7m docs per shard. Could the cause for latency be boost > queries and would it also be time spent in aggregating results from many > shards? Curious to learn more about caching such/short queries that @Mikhail > mentioned. > >> On Tue, Apr 18, 2023 at 9:44 PM Dave <hastings.recurs...@gmail.com> wrote: >> >> I think there are more important questions here. What do you want with a >> *:* query? Do you want all the results in on return? Or do you just want >> the count of total documents? Or to put the results in facets? *:* should >> never take long unless you are requesting every single document not just >> the first ten. >> >>> On Apr 18, 2023, at 9:05 PM, Rajani Maski <rajinima...@gmail.com> wrote: >>> >>> Hi Mikhail, >>> >>> Yes, 9.1.1, that should be helpful, can you please point me to the >>> related jira(s) and/or docs? >>> >>> Thank you, >>> Rajani >>> >>> >>> >>>> On Mon, Apr 17, 2023 at 2:09 AM Mikhail Khludnev <m...@apache.org> >> wrote: >>>> >>>> Hello Rajani. >>>> Which version are you running? IIRC 9.1.2 has some >>>> improvement about caching short queries. >>>> >>>> On Sun, Apr 16, 2023 at 4:25 PM Rajani Maski <rajinima...@gmail.com> >>>> wrote: >>>> >>>>> Hi Solr Users, >>>>> >>>>> What are your suggestions to improve star queries latencies? By star >>>>> queries I mean "*:*" or single term queries having boost formulas >> (such >>>> as >>>>> doc recency and many others) taking 10 or more seconds. It is a large >>>>> collection with good compute resources, however I am guessing this may >> be >>>>> because each shard has too many documents and I noticed per shard >>>> response >>>>> time also is high. >>>>> >>>>> Splitting shards could be an option however it is already an >>>>> evenly distributed, composite router, 96 shards collection, I am >>>>> concerned that more than 100 shards per collection can lead to >>>> exhaustively >>>>> searching too many shards and aggregation issues. What are your >> thoughts? >>>>> >>>>> Can we make use of any caches, query result cache or other caches, in >>>> solr >>>>> that allows warming up and persisting these queries results in ram, and >>>>> that maybe helps reduce this query time? >>>>> >>>>> Thanks, >>>>> Rajani >>>>> >>>> >>>> >>>> -- >>>> Sincerely yours >>>> Mikhail Khludnev >>>> https://t.me/MUST_SEARCH >>>> A caveat: Cyrillic! >>>> >>
