Clarifications of minimum should match parameter and optional clauses in eDisMax

Hi everyone, I have this query that uses the Boolean Query Parser: *http://localhost:8983/solr/books/select ?* *q={!bool filter=$**categoryIds** should=$queryTerms}&* *categoryIds**=**category:(1 2 5 7)**&* *queryTerms=**content**:(user text)* I would like

Re: Solr query using full heap and triggers stop the world pause

On 1/3/24 13:33, rajani m wrote: Solr query with LTR as a re-ranker is using full heap all of sudden and triggering STW pause. Could you please take a look and let me know your thoughts? What is causing this? The STW is putting nodes in an unhealthy state causing nodes to restart and bringi

Re: Solr query using full heap and triggers stop the world pause

Thank you Shawn, that was very helpful. I have tried the G1HeapRegionSize setting. I set it to 32m(XX:G1HeapRegionSize=32m) and replayed the same query logs, but it didn't help, reproducing the same oom error. I was able to capture the heap dump when the heap was almost full and have the heap ana

Re: Solr query using full heap and triggers stop the world pause

reRankDocs is set to 1000. I would try with a lower number, like 100. If the best match is not in the top 100 documents, something is wrong with the base relevance algorithm. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jan 4, 2024, at 9:28 AM, r

Re: Solr query using full heap and triggers stop the world pause

Hi Wunder, The base ranker takes care of matching and ranking docs based on qf, pf2 and pf3, the ltr re-ranker looks at bunch of user behavior fields/features such as date(recency), popularity, favorited, saves and hence reranking 1k presents better quality than top 100. Thanks, Rajani On Thu

Re: Solr query using full heap and triggers stop the world pause

Try it with 100 and see if it runs out of heap. It if does not run out, then the size of reRankDocs is the cause. You can increase the heap if you want to, but if the reranker is moving document 1000 places in the result list, I would look seriously at improving the base relevance. You might in

ExitableDirectoryReader warning

Hi, What scenarios would solr warn about "ExitableDirectoryReader" ? I see several "ExitableDirectoryReader$ExitingReaderException: The request took too long to iterate over terms.". What causes this this? Is it a shard size issue or lack of any resources, heap, ram or cpu? . Log snippets -

Re: Optimize latency of queries with facets

How does it work with facet.method=enum? Slicing to even smaller shards makes sense. On Wed, Jan 3, 2024 at 12:10 AM rajani m wrote: > Hi Solr Users, > >Queries with facets show 2x increase in latency. This facet field is a > boolean field with docvalues enabled[1]. Same queries without fac

Re: ExitableDirectoryReader warning

It might be any or all of these reasons. On Thu, Jan 4, 2024 at 10:18 PM rajani m wrote: > Hi, > > What scenarios would solr warn about "ExitableDirectoryReader" ? I see > several "ExitableDirectoryReader$ExitingReaderException: The request took > too long to iterate over terms.". What cause

Re: ExitableDirectoryReader warning

Hi Mikhail, Did you intend to include a list of reasons? I only got "It might be any or all of these reasons." Thank you, Rajani On Thu, Jan 4, 2024 at 3:20 PM Mikhail Khludnev wrote: > It might be any or all of these reasons. > > On Thu, Jan 4, 2024 at 10:18 PM rajani m wrote: > > > Hi, > >

Re: ExitableDirectoryReader warning

> Is it a shard size issue or lack of any resources, heap, ram or cpu? On Thu, Jan 4, 2024 at 11:32 PM rajani m wrote: > Hi Mikhail, > Did you intend to include a list of reasons? I only got "It might be any > or all of these reasons." > > Thank you, > Rajani > > On Thu, Jan 4, 2024 at 3:20 

Re: Clarifications of minimum should match parameter and optional clauses in eDisMax

Hello Anna. Agreed. It's a little bit counterintuitive, but described explicitly in the braced note here https://lucene.apache.org/core/9_7_0/core/org/apache/lucene/search/BooleanQuery.Builder.html#setMinimumNumberShouldMatch(int) To achieve what you need > set to 0, meaning that all clauses are

Re: ExitableDirectoryReader warning

This is the ramification of using "timeAllowed" (a request param to a search) ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Jan 4, 2024 at 2:18 PM rajani m wrote: > Hi, > > What scenarios would solr warn about "ExitableDirectoryReader" ?