Clarifications of minimum should match parameter and optional clauses in eDisMax

[Anna Ruggero]

Hi everyone,

I have this query that uses the Boolean Query Parser:
*http://localhost:8983/solr/books/select
?*
*q={!bool filter=$**categoryIds** should=$queryTerms}&*
*categoryIds**=**category:(1 2 5 7)**&*
*queryTerms=**content**:(user text)*

I would like to achieve the same result using the eDisMax query parser, but
I encountered some problems with the minimum should match clause.

Here is the query:
*http://localhost:8983/solr/books/select
?*
*fq=category:(1 2 5 7)&*
*q=user text&*
*df=content&*
*defType=edismax*

It returns only documents having "*user"* or "*text"* in the content field
(but at least one of them is needed). If no documents contain those words
in the *content* field, I have zero results returned.
>From the debug, I could see that the parsed query is:
*+(DisjunctionMaxQuery((**content**:user)) DisjunctionMaxQuery((**content*
*:text)))*

By default the minimum should match parameter is set to 0, meaning that all
clauses are optional. Why is there the + operator before the two
disjunction queries? I wouldn't expect to see it. I would expect to see all
documents returned even if none of them match any term in q.

Also, if I explicitly set mm to 1 like:
*http://localhost:8983/solr/books/select
?*
*fq=category:(1 2 5 7)&*
*q=user text&*
*df=content&*
*defType=edismax&*
*mm=1*

The parsed query is:
*+(DisjunctionMaxQuery((**content**:**user**)) DisjunctionMaxQuery((*
*content**:**text*
*)))~1*
Whose meaning is the same as the query before (with mm=0).

What is the difference between mm=0 and mm=1?


Thank you!

*Anna Ruggero*
R&D Machine Learning / Software Engineer, Search Consultant
e-mail: a.rugg...@sease.io


*Sease* - Information Retrieval Applied
Consulting | Training | Open Source
Sease.io 
LinkedIn  | Twitter
 | Youtube
 | Github

Re: Solr query using full heap and triggers stop the world pause

[Shawn Heisey]

On 1/3/24 13:33, rajani m wrote:

 Solr query with LTR as a re-ranker is using full heap all of sudden and
triggering STW pause. Could you please take a look and let me know your
thoughts? What is causing this? The STW  is putting nodes in an unhealthy
state causing nodes to restart and bringing the entire cluster down.

As per logs, the issue seems to be related to LTR generating features at
query time. The model has 12 features and most features are solr query and
few field values. The error from the logs is copied below[2].  I'd say this
is a major bug as G1GC is supposed to avoid STW.  What are your thoughts?


G1 does not completely eliminate stop-the-world.

One of the little details of G1GC operation concerns something called 
humongous objects.


Any object larger than half the G1 region size is classified as 
humongous.  These objects are allocated directly in the old region, and 
the only way they can be collected is during a full garbage collection.


The secret to stellar performance with G1 is to eliminate, as much as 
possible, full GC cycles ... because there will always be a long STW 
with a full G1GC, but G1's region-specific collectors operate almost 
entirely concurrently with the application.


You can set the G1 region size with the `-XX:G1HeapRegionSize` parameter 
in your GC tuning ... but be aware that the max region size is 32m. 
Which means that no matter what when using G1, an object that is 16 
megabytes or larger will always be humongous.  It is my understanding 
that LTR models can be many megabytes in size, but I have never used the 
feature myself.


If you are running on Java 11 or later, I recommend giving ZGC a try. 
This is the tuning I use in /etc/default/solr.in.sh.  I use OpenJDK 17:


GC_TUNE=" \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseZGC \
  -XX:+ParallelRefProcEnabled \
  -XX:+ExplicitGCInvokesConcurrent \
  -XX:+AlwaysPreTouch \
  -XX:+UseNUMA \
"

ZGC promises extremely short GC pauses with ANY size heap, even 
terabytes.  I haven't tested it with a large heap myself, but in my 
limited testing, its individual pauses were MUCH shorter than what I saw 
with G1.  Throughput is lower than G1, but latency is AWESOME.


One bit of warning ... ZGC always uses 64-bit pointers, so the advice 
you'll commonly see recommending a heap size below 32GB does not apply 
to ZGC.  There is no advantage to a 31GB heap compared to 32GB when 
using ZGC.


Thanks,
Shawn

Re: Solr query using full heap and triggers stop the world pause

[rajani m]

Thank you Shawn, that was very helpful.  I have tried the G1HeapRegionSize
setting. I set it to 32m(XX:G1HeapRegionSize=32m) and replayed the same
query logs, but it didn't help, reproducing the same oom error.

I was able to capture the heap dump when the heap was almost full and have
the heap analysis report generated by MAT, uploaded here on my drive
.
Whenever you can, could you please take a look and let me know your
thoughts? Although the issue is reproducible only when the query has LTR as
reranker, the core issue seems to be originating from main libraries is
what report seems to be implying. Let me know what you think.

I will test with ZGC and see if it can prevent STW, old generation full gc,
will let you know.

Thanks,
Rajani


On Thu, Jan 4, 2024 at 11:20 AM Shawn Heisey 
wrote:

> On 1/3/24 13:33, rajani m wrote:
> >  Solr query with LTR as a re-ranker is using full heap all of sudden
> and
> > triggering STW pause. Could you please take a look and let me know your
> > thoughts? What is causing this? The STW  is putting nodes in an unhealthy
> > state causing nodes to restart and bringing the entire cluster down.
> >
> > As per logs, the issue seems to be related to LTR generating features at
> > query time. The model has 12 features and most features are solr query
> and
> > few field values. The error from the logs is copied below[2].  I'd say
> this
> > is a major bug as G1GC is supposed to avoid STW.  What are your thoughts?
>
> G1 does not completely eliminate stop-the-world.
>
> One of the little details of G1GC operation concerns something called
> humongous objects.
>
> Any object larger than half the G1 region size is classified as
> humongous.  These objects are allocated directly in the old region, and
> the only way they can be collected is during a full garbage collection.
>
> The secret to stellar performance with G1 is to eliminate, as much as
> possible, full GC cycles ... because there will always be a long STW
> with a full G1GC, but G1's region-specific collectors operate almost
> entirely concurrently with the application.
>
> You can set the G1 region size with the `-XX:G1HeapRegionSize` parameter
> in your GC tuning ... but be aware that the max region size is 32m.
> Which means that no matter what when using G1, an object that is 16
> megabytes or larger will always be humongous.  It is my understanding
> that LTR models can be many megabytes in size, but I have never used the
> feature myself.
>
> If you are running on Java 11 or later, I recommend giving ZGC a try.
> This is the tuning I use in /etc/default/solr.in.sh.  I use OpenJDK 17:
>
> GC_TUNE=" \
>-XX:+UnlockExperimentalVMOptions \
>-XX:+UseZGC \
>-XX:+ParallelRefProcEnabled \
>-XX:+ExplicitGCInvokesConcurrent \
>-XX:+AlwaysPreTouch \
>-XX:+UseNUMA \
> "
>
> ZGC promises extremely short GC pauses with ANY size heap, even
> terabytes.  I haven't tested it with a large heap myself, but in my
> limited testing, its individual pauses were MUCH shorter than what I saw
> with G1.  Throughput is lower than G1, but latency is AWESOME.
>
> One bit of warning ... ZGC always uses 64-bit pointers, so the advice
> you'll commonly see recommending a heap size below 32GB does not apply
> to ZGC.  There is no advantage to a 31GB heap compared to 32GB when
> using ZGC.
>
> Thanks,
> Shawn
>
>

Re: Solr query using full heap and triggers stop the world pause

[Walter Underwood]

reRankDocs is set to 1000. I would try with a lower number, like 100. If the 
best match is not in the top 100 documents, something is wrong with the base 
relevance algorithm.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jan 4, 2024, at 9:28 AM, rajani m  wrote:
> 
> Thank you Shawn, that was very helpful.  I have tried the G1HeapRegionSize
> setting. I set it to 32m(XX:G1HeapRegionSize=32m) and replayed the same
> query logs, but it didn't help, reproducing the same oom error.
> 
> I was able to capture the heap dump when the heap was almost full and have
> the heap analysis report generated by MAT, uploaded here on my drive
> .
> Whenever you can, could you please take a look and let me know your
> thoughts? Although the issue is reproducible only when the query has LTR as
> reranker, the core issue seems to be originating from main libraries is
> what report seems to be implying. Let me know what you think.
> 
> I will test with ZGC and see if it can prevent STW, old generation full gc,
> will let you know.
> 
> Thanks,
> Rajani
> 
> 
> On Thu, Jan 4, 2024 at 11:20 AM Shawn Heisey 
> wrote:
> 
>> On 1/3/24 13:33, rajani m wrote:
>>> Solr query with LTR as a re-ranker is using full heap all of sudden
>> and
>>> triggering STW pause. Could you please take a look and let me know your
>>> thoughts? What is causing this? The STW  is putting nodes in an unhealthy
>>> state causing nodes to restart and bringing the entire cluster down.
>>> 
>>> As per logs, the issue seems to be related to LTR generating features at
>>> query time. The model has 12 features and most features are solr query
>> and
>>> few field values. The error from the logs is copied below[2].  I'd say
>> this
>>> is a major bug as G1GC is supposed to avoid STW.  What are your thoughts?
>> 
>> G1 does not completely eliminate stop-the-world.
>> 
>> One of the little details of G1GC operation concerns something called
>> humongous objects.
>> 
>> Any object larger than half the G1 region size is classified as
>> humongous.  These objects are allocated directly in the old region, and
>> the only way they can be collected is during a full garbage collection.
>> 
>> The secret to stellar performance with G1 is to eliminate, as much as
>> possible, full GC cycles ... because there will always be a long STW
>> with a full G1GC, but G1's region-specific collectors operate almost
>> entirely concurrently with the application.
>> 
>> You can set the G1 region size with the `-XX:G1HeapRegionSize` parameter
>> in your GC tuning ... but be aware that the max region size is 32m.
>> Which means that no matter what when using G1, an object that is 16
>> megabytes or larger will always be humongous.  It is my understanding
>> that LTR models can be many megabytes in size, but I have never used the
>> feature myself.
>> 
>> If you are running on Java 11 or later, I recommend giving ZGC a try.
>> This is the tuning I use in /etc/default/solr.in.sh.  I use OpenJDK 17:
>> 
>> GC_TUNE=" \
>>   -XX:+UnlockExperimentalVMOptions \
>>   -XX:+UseZGC \
>>   -XX:+ParallelRefProcEnabled \
>>   -XX:+ExplicitGCInvokesConcurrent \
>>   -XX:+AlwaysPreTouch \
>>   -XX:+UseNUMA \
>> "
>> 
>> ZGC promises extremely short GC pauses with ANY size heap, even
>> terabytes.  I haven't tested it with a large heap myself, but in my
>> limited testing, its individual pauses were MUCH shorter than what I saw
>> with G1.  Throughput is lower than G1, but latency is AWESOME.
>> 
>> One bit of warning ... ZGC always uses 64-bit pointers, so the advice
>> you'll commonly see recommending a heap size below 32GB does not apply
>> to ZGC.  There is no advantage to a 31GB heap compared to 32GB when
>> using ZGC.
>> 
>> Thanks,
>> Shawn
>> 
>> 

Re: Solr query using full heap and triggers stop the world pause

[rajani m]

Hi Wunder,

  The base ranker takes care of matching and ranking docs based on qf, pf2
and pf3, the ltr re-ranker looks at bunch of user behavior fields/features
such as date(recency), popularity, favorited, saves and hence reranking 1k
presents better quality than top 100.


Thanks,
Rajani

On Thu, Jan 4, 2024 at 12:33 PM Walter Underwood 
wrote:

> reRankDocs is set to 1000. I would try with a lower number, like 100. If
> the best match is not in the top 100 documents, something is wrong with the
> base relevance algorithm.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> > On Jan 4, 2024, at 9:28 AM, rajani m  wrote:
> >
> > Thank you Shawn, that was very helpful.  I have tried the
> G1HeapRegionSize
> > setting. I set it to 32m(XX:G1HeapRegionSize=32m) and replayed the same
> > query logs, but it didn't help, reproducing the same oom error.
> >
> > I was able to capture the heap dump when the heap was almost full and
> have
> > the heap analysis report generated by MAT, uploaded here on my drive
> > <
> https://drive.google.com/file/d/1j1ghQB-zezTu8dje5pWJQE5A0qzyZ1ro/view?usp=sharing
> >.
> > Whenever you can, could you please take a look and let me know your
> > thoughts? Although the issue is reproducible only when the query has LTR
> as
> > reranker, the core issue seems to be originating from main libraries is
> > what report seems to be implying. Let me know what you think.
> >
> > I will test with ZGC and see if it can prevent STW, old generation full
> gc,
> > will let you know.
> >
> > Thanks,
> > Rajani
> >
> >
> > On Thu, Jan 4, 2024 at 11:20 AM Shawn Heisey  >
> > wrote:
> >
> >> On 1/3/24 13:33, rajani m wrote:
> >>> Solr query with LTR as a re-ranker is using full heap all of sudden
> >> and
> >>> triggering STW pause. Could you please take a look and let me know your
> >>> thoughts? What is causing this? The STW  is putting nodes in an
> unhealthy
> >>> state causing nodes to restart and bringing the entire cluster down.
> >>>
> >>> As per logs, the issue seems to be related to LTR generating features
> at
> >>> query time. The model has 12 features and most features are solr query
> >> and
> >>> few field values. The error from the logs is copied below[2].  I'd say
> >> this
> >>> is a major bug as G1GC is supposed to avoid STW.  What are your
> thoughts?
> >>
> >> G1 does not completely eliminate stop-the-world.
> >>
> >> One of the little details of G1GC operation concerns something called
> >> humongous objects.
> >>
> >> Any object larger than half the G1 region size is classified as
> >> humongous.  These objects are allocated directly in the old region, and
> >> the only way they can be collected is during a full garbage collection.
> >>
> >> The secret to stellar performance with G1 is to eliminate, as much as
> >> possible, full GC cycles ... because there will always be a long STW
> >> with a full G1GC, but G1's region-specific collectors operate almost
> >> entirely concurrently with the application.
> >>
> >> You can set the G1 region size with the `-XX:G1HeapRegionSize` parameter
> >> in your GC tuning ... but be aware that the max region size is 32m.
> >> Which means that no matter what when using G1, an object that is 16
> >> megabytes or larger will always be humongous.  It is my understanding
> >> that LTR models can be many megabytes in size, but I have never used the
> >> feature myself.
> >>
> >> If you are running on Java 11 or later, I recommend giving ZGC a try.
> >> This is the tuning I use in /etc/default/solr.in.sh.  I use OpenJDK 17:
> >>
> >> GC_TUNE=" \
> >>   -XX:+UnlockExperimentalVMOptions \
> >>   -XX:+UseZGC \
> >>   -XX:+ParallelRefProcEnabled \
> >>   -XX:+ExplicitGCInvokesConcurrent \
> >>   -XX:+AlwaysPreTouch \
> >>   -XX:+UseNUMA \
> >> "
> >>
> >> ZGC promises extremely short GC pauses with ANY size heap, even
> >> terabytes.  I haven't tested it with a large heap myself, but in my
> >> limited testing, its individual pauses were MUCH shorter than what I saw
> >> with G1.  Throughput is lower than G1, but latency is AWESOME.
> >>
> >> One bit of warning ... ZGC always uses 64-bit pointers, so the advice
> >> you'll commonly see recommending a heap size below 32GB does not apply
> >> to ZGC.  There is no advantage to a 31GB heap compared to 32GB when
> >> using ZGC.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
>
>

Re: Solr query using full heap and triggers stop the world pause

[Walter Underwood]

Try it with 100 and see if it runs out of heap. It if does not run out, then 
the size of reRankDocs is the cause.

You can increase the heap if you want to, but if the reranker is moving 
document 1000 places in the result list, I would look seriously at improving 
the base relevance. You might include an aggregate popularity, for example. 
Maybe add overall recency.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jan 4, 2024, at 10:28 AM, rajani m  wrote:
> 
> Hi Wunder,
> 
>  The base ranker takes care of matching and ranking docs based on qf, pf2
> and pf3, the ltr re-ranker looks at bunch of user behavior fields/features
> such as date(recency), popularity, favorited, saves and hence reranking 1k
> presents better quality than top 100.
> 
> 
> Thanks,
> Rajani
> 
> On Thu, Jan 4, 2024 at 12:33 PM Walter Underwood 
> wrote:
> 
>> reRankDocs is set to 1000. I would try with a lower number, like 100. If
>> the best match is not in the top 100 documents, something is wrong with the
>> base relevance algorithm.
>> 
>> wunder
>> Walter Underwood
>> wun...@wunderwood.org
>> http://observer.wunderwood.org/  (my blog)
>> 
>>> On Jan 4, 2024, at 9:28 AM, rajani m  wrote:
>>> 
>>> Thank you Shawn, that was very helpful.  I have tried the
>> G1HeapRegionSize
>>> setting. I set it to 32m(XX:G1HeapRegionSize=32m) and replayed the same
>>> query logs, but it didn't help, reproducing the same oom error.
>>> 
>>> I was able to capture the heap dump when the heap was almost full and
>> have
>>> the heap analysis report generated by MAT, uploaded here on my drive
>>> <
>> https://drive.google.com/file/d/1j1ghQB-zezTu8dje5pWJQE5A0qzyZ1ro/view?usp=sharing
>>> .
>>> Whenever you can, could you please take a look and let me know your
>>> thoughts? Although the issue is reproducible only when the query has LTR
>> as
>>> reranker, the core issue seems to be originating from main libraries is
>>> what report seems to be implying. Let me know what you think.
>>> 
>>> I will test with ZGC and see if it can prevent STW, old generation full
>> gc,
>>> will let you know.
>>> 
>>> Thanks,
>>> Rajani
>>> 
>>> 
>>> On Thu, Jan 4, 2024 at 11:20 AM Shawn Heisey >> 
>>> wrote:
>>> 
 On 1/3/24 13:33, rajani m wrote:
>Solr query with LTR as a re-ranker is using full heap all of sudden
 and
> triggering STW pause. Could you please take a look and let me know your
> thoughts? What is causing this? The STW  is putting nodes in an
>> unhealthy
> state causing nodes to restart and bringing the entire cluster down.
> 
> As per logs, the issue seems to be related to LTR generating features
>> at
> query time. The model has 12 features and most features are solr query
 and
> few field values. The error from the logs is copied below[2].  I'd say
 this
> is a major bug as G1GC is supposed to avoid STW.  What are your
>> thoughts?
 
 G1 does not completely eliminate stop-the-world.
 
 One of the little details of G1GC operation concerns something called
 humongous objects.
 
 Any object larger than half the G1 region size is classified as
 humongous.  These objects are allocated directly in the old region, and
 the only way they can be collected is during a full garbage collection.
 
 The secret to stellar performance with G1 is to eliminate, as much as
 possible, full GC cycles ... because there will always be a long STW
 with a full G1GC, but G1's region-specific collectors operate almost
 entirely concurrently with the application.
 
 You can set the G1 region size with the `-XX:G1HeapRegionSize` parameter
 in your GC tuning ... but be aware that the max region size is 32m.
 Which means that no matter what when using G1, an object that is 16
 megabytes or larger will always be humongous.  It is my understanding
 that LTR models can be many megabytes in size, but I have never used the
 feature myself.
 
 If you are running on Java 11 or later, I recommend giving ZGC a try.
 This is the tuning I use in /etc/default/solr.in.sh.  I use OpenJDK 17:
 
 GC_TUNE=" \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseZGC \
  -XX:+ParallelRefProcEnabled \
  -XX:+ExplicitGCInvokesConcurrent \
  -XX:+AlwaysPreTouch \
  -XX:+UseNUMA \
 "
 
 ZGC promises extremely short GC pauses with ANY size heap, even
 terabytes.  I haven't tested it with a large heap myself, but in my
 limited testing, its individual pauses were MUCH shorter than what I saw
 with G1.  Throughput is lower than G1, but latency is AWESOME.
 
 One bit of warning ... ZGC always uses 64-bit pointers, so the advice
 you'll commonly see recommending a heap size below 32GB does not apply
 to ZGC.  There is no advantage to a 31GB heap compared to 32GB when
 using ZGC.
 
 Thanks,
 Shawn
 
 
>> 
>> 

ExitableDirectoryReader warning

[rajani m]

Hi,

What scenarios would solr warn about "ExitableDirectoryReader" ? I see
several "ExitableDirectoryReader$ExitingReaderException: The request took
too long to iterate over terms.". What causes this this? Is it a shard size
issue or lack of any resources, heap, ram or cpu?  .

Log snippets -

org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
The request took too long to iterate over terms. Timeout: timeoutAt:
2101774016440850 (System.nanoTime(): 2101774068035377),
TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@79b2db56
at 
org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.checkTimeoutWithSampling(ExitableDirectoryReader.java:749)
at 
org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.(ExitableDirectoryReader.java:737)
at 
org.apache.lucene.index.ExitableDirectoryReader$ExitableTerms.iterator(ExitableDirectoryReader.java:719)
at org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:116)
at org.apache.lucene.index.TermStates.build(TermStates.java:102)
at org.apache.lucene.search.PhraseQuery$1.getStats(PhraseQuery.java:461)
at org.apache.lucene.search.PhraseWeight.(PhraseWeight.java:44)
at org.apache.lucene.search.PhraseQuery$1.(PhraseQuery.java:441)
at 
org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:441)
at org.apache.lucene.search.BoostQuery.createWeight(BoostQuery.java:124)
at 
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:869)
at org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:59)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:237)
at 
org.apache.lucene.queries.function.FunctionScoreQuery.createWeight(FunctionScoreQuery.java:113)


(from a different time window)


org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
The request took too long to iterate over doc values. Timeout:
timeoutAt: 2103004456482731 (System.nanoTime(): 2103004484203805),
DocValues=org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$2@2e1600c4
at 
org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:394)
at 
org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader$1.advance(ExitableDirectoryReader.java:124)
at 
org.apache.lucene.queries.function.valuesource.IntFieldSource$1.exists(IntFieldSource.java:81)
at 
org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:39)
at 
org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
at 
org.apache.lucene.queries.function.valuesource.SumFloatFunction.func(SumFloatFunction.java:38)
at 
org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
at 
org.apache.lucene.queries.function.docvalues.FloatDocValues.doubleVal(FloatDocValues.java:66)
at 
org.apache.solr.search.ValueSourceParser$33.func(ValueSourceParser.java:587)
at 
org.apache.solr.search.ValueSourceParser$DoubleParser$Function$1.doubleVal(ValueSourceParser.java:1666)
at 
org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:51)
at 
org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:38)
at 
org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
at 
org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:126)
at 
org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:41)

Re: Optimize latency of queries with facets

[Mikhail Khludnev]

How does it work with facet.method=enum?
Slicing to even smaller shards makes sense.

On Wed, Jan 3, 2024 at 12:10 AM rajani m  wrote:

> Hi Solr Users,
>
>Queries with facets show 2x increase in latency. This facet field is a
> boolean field with docvalues enabled[1].  Same queries without facets have
> ~50 ms and with facets it is ~120ms.  Is this an expected performance?
> Appreciate any suggestions to optimize it.
>
> Additional details - I am querying a single shard,
> distrib=false&cache=false, standard query such as
> "&q=cats&defType=edismax&qf=keywords title desc body" have ~50 ms qtime and
> with facets (facet=true&facet.field=has_color) it is jumping to 120ms. Is
> this expected performance with facets, or are there other factors that
> could be cause result to such increase in the latency? Could this be
> because of shard index size which is 65 gb, is this big? and would
> splitting the shard into a 30gb size improve the performance? Cpu and ram
> should not be a concern, the node hosting shard has enough ram to fit the
> entire shard index and has 16vcpus. What other factors can result in
> increasing latency? Appreciate suggestions to improve the latency.
>
> [1]  omitNorms="true" docValues="true"/>
>
> Thanks,
> Rajani
>


-- 
Sincerely yours
Mikhail Khludnev

Re: ExitableDirectoryReader warning

[Mikhail Khludnev]

It might be any or all of these reasons.

On Thu, Jan 4, 2024 at 10:18 PM rajani m  wrote:

> Hi,
>
> What scenarios would solr warn about "ExitableDirectoryReader" ? I see
> several "ExitableDirectoryReader$ExitingReaderException: The request took
> too long to iterate over terms.". What causes this this? Is it a shard size
> issue or lack of any resources, heap, ram or cpu?  .
>
> Log snippets -
>
> org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> The request took too long to iterate over terms. Timeout: timeoutAt:
> 2101774016440850 (System.nanoTime(): 2101774068035377),
>
> TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@79b2db56
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.checkTimeoutWithSampling(ExitableDirectoryReader.java:749)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.(ExitableDirectoryReader.java:737)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTerms.iterator(ExitableDirectoryReader.java:719)
> at
> org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:116)
> at org.apache.lucene.index.TermStates.build(TermStates.java:102)
> at
> org.apache.lucene.search.PhraseQuery$1.getStats(PhraseQuery.java:461)
> at
> org.apache.lucene.search.PhraseWeight.(PhraseWeight.java:44)
> at
> org.apache.lucene.search.PhraseQuery$1.(PhraseQuery.java:441)
> at
> org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:441)
> at
> org.apache.lucene.search.BoostQuery.createWeight(BoostQuery.java:124)
> at
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:869)
> at
> org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:59)
> at
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:237)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery.createWeight(FunctionScoreQuery.java:113)
>
>
> (from a different time window)
>
>
> org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> The request took too long to iterate over doc values. Timeout:
> timeoutAt: 2103004456482731 (System.nanoTime(): 2103004484203805),
>
> DocValues=org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$2@2e1600c4
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:394)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader$1.advance(ExitableDirectoryReader.java:124)
> at
> org.apache.lucene.queries.function.valuesource.IntFieldSource$1.exists(IntFieldSource.java:81)
> at
> org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:39)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.valuesource.SumFloatFunction.func(SumFloatFunction.java:38)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.docvalues.FloatDocValues.doubleVal(FloatDocValues.java:66)
> at
> org.apache.solr.search.ValueSourceParser$33.func(ValueSourceParser.java:587)
> at
> org.apache.solr.search.ValueSourceParser$DoubleParser$Function$1.doubleVal(ValueSourceParser.java:1666)
> at
> org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:51)
> at
> org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:38)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:126)
> at
> org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:41)
>


-- 
Sincerely yours
Mikhail Khludnev

Re: ExitableDirectoryReader warning

[rajani m]

Hi Mikhail,
 Did you intend to include a list of reasons? I only got  "It might be any
or all of these reasons."

Thank you,
Rajani

On Thu, Jan 4, 2024 at 3:20 PM Mikhail Khludnev  wrote:

> It might be any or all of these reasons.
>
> On Thu, Jan 4, 2024 at 10:18 PM rajani m  wrote:
>
> > Hi,
> >
> > What scenarios would solr warn about "ExitableDirectoryReader" ? I
> see
> > several "ExitableDirectoryReader$ExitingReaderException: The request took
> > too long to iterate over terms.". What causes this this? Is it a shard
> size
> > issue or lack of any resources, heap, ram or cpu?  .
> >
> > Log snippets -
> >
> > org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> > The request took too long to iterate over terms. Timeout: timeoutAt:
> > 2101774016440850 (System.nanoTime(): 2101774068035377),
> >
> >
> TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@79b2db56
> > at
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.checkTimeoutWithSampling(ExitableDirectoryReader.java:749)
> > at
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.(ExitableDirectoryReader.java:737)
> > at
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTerms.iterator(ExitableDirectoryReader.java:719)
> > at
> > org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:116)
> > at org.apache.lucene.index.TermStates.build(TermStates.java:102)
> > at
> > org.apache.lucene.search.PhraseQuery$1.getStats(PhraseQuery.java:461)
> > at
> > org.apache.lucene.search.PhraseWeight.(PhraseWeight.java:44)
> > at
> > org.apache.lucene.search.PhraseQuery$1.(PhraseQuery.java:441)
> > at
> > org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:441)
> > at
> > org.apache.lucene.search.BoostQuery.createWeight(BoostQuery.java:124)
> > at
> >
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:869)
> > at
> > org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:59)
> > at
> > org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:237)
> > at
> >
> org.apache.lucene.queries.function.FunctionScoreQuery.createWeight(FunctionScoreQuery.java:113)
> >
> >
> > (from a different time window)
> >
> >
> > org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> > The request took too long to iterate over doc values. Timeout:
> > timeoutAt: 2103004456482731 (System.nanoTime(): 2103004484203805),
> >
> >
> DocValues=org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$2@2e1600c4
> > at
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:394)
> > at
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader$1.advance(ExitableDirectoryReader.java:124)
> > at
> >
> org.apache.lucene.queries.function.valuesource.IntFieldSource$1.exists(IntFieldSource.java:81)
> > at
> >
> org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:39)
> > at
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > at
> >
> org.apache.lucene.queries.function.valuesource.SumFloatFunction.func(SumFloatFunction.java:38)
> > at
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > at
> >
> org.apache.lucene.queries.function.docvalues.FloatDocValues.doubleVal(FloatDocValues.java:66)
> > at
> >
> org.apache.solr.search.ValueSourceParser$33.func(ValueSourceParser.java:587)
> > at
> >
> org.apache.solr.search.ValueSourceParser$DoubleParser$Function$1.doubleVal(ValueSourceParser.java:1666)
> > at
> >
> org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:51)
> > at
> >
> org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:38)
> > at
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > at
> >
> org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:126)
> > at
> >
> org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:41)
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>

Re: ExitableDirectoryReader warning

[Mikhail Khludnev]

>   Is it a shard size
issue or lack of any resources, heap, ram or cpu?

On Thu, Jan 4, 2024 at 11:32 PM rajani m  wrote:

> Hi Mikhail,
>  Did you intend to include a list of reasons? I only got  "It might be any
> or all of these reasons."
>
> Thank you,
> Rajani
>
> On Thu, Jan 4, 2024 at 3:20 PM Mikhail Khludnev  wrote:
>
> > It might be any or all of these reasons.
> >
> > On Thu, Jan 4, 2024 at 10:18 PM rajani m  wrote:
> >
> > > Hi,
> > >
> > > What scenarios would solr warn about "ExitableDirectoryReader" ? I
> > see
> > > several "ExitableDirectoryReader$ExitingReaderException: The request
> took
> > > too long to iterate over terms.". What causes this this? Is it a shard
> > size
> > > issue or lack of any resources, heap, ram or cpu?  .
> > >
> > > Log snippets -
> > >
> > > org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> > > The request took too long to iterate over terms. Timeout: timeoutAt:
> > > 2101774016440850 (System.nanoTime(): 2101774068035377),
> > >
> > >
> >
> TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@79b2db56
> > > at
> > >
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.checkTimeoutWithSampling(ExitableDirectoryReader.java:749)
> > > at
> > >
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.(ExitableDirectoryReader.java:737)
> > > at
> > >
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTerms.iterator(ExitableDirectoryReader.java:719)
> > > at
> > > org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:116)
> > > at
> org.apache.lucene.index.TermStates.build(TermStates.java:102)
> > > at
> > > org.apache.lucene.search.PhraseQuery$1.getStats(PhraseQuery.java:461)
> > > at
> > > org.apache.lucene.search.PhraseWeight.(PhraseWeight.java:44)
> > > at
> > > org.apache.lucene.search.PhraseQuery$1.(PhraseQuery.java:441)
> > > at
> > > org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:441)
> > > at
> > > org.apache.lucene.search.BoostQuery.createWeight(BoostQuery.java:124)
> > > at
> > >
> >
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:869)
> > > at
> > > org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:59)
> > > at
> > >
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:237)
> > > at
> > >
> >
> org.apache.lucene.queries.function.FunctionScoreQuery.createWeight(FunctionScoreQuery.java:113)
> > >
> > >
> > > (from a different time window)
> > >
> > >
> > > org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> > > The request took too long to iterate over doc values. Timeout:
> > > timeoutAt: 2103004456482731 (System.nanoTime(): 2103004484203805),
> > >
> > >
> >
> DocValues=org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$2@2e1600c4
> > > at
> > >
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:394)
> > > at
> > >
> >
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader$1.advance(ExitableDirectoryReader.java:124)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.IntFieldSource$1.exists(IntFieldSource.java:81)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:39)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.SumFloatFunction.func(SumFloatFunction.java:38)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > > at
> > >
> >
> org.apache.lucene.queries.function.docvalues.FloatDocValues.doubleVal(FloatDocValues.java:66)
> > > at
> > >
> >
> org.apache.solr.search.ValueSourceParser$33.func(ValueSourceParser.java:587)
> > > at
> > >
> >
> org.apache.solr.search.ValueSourceParser$DoubleParser$Function$1.doubleVal(ValueSourceParser.java:1666)
> > > at
> > >
> >
> org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:51)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:38)
> > > at
> > >
> >
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> > > at
> > >
> >
> org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:126)
> > > at
> > >
> >
> org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:41)
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Clarifications of minimum should match parameter and optional clauses in eDisMax

[Mikhail Khludnev]

Hello Anna.
Agreed. It's a little bit counterintuitive, but described explicitly in the
braced note here
https://lucene.apache.org/core/9_7_0/core/org/apache/lucene/search/BooleanQuery.Builder.html#setMinimumNumberShouldMatch(int)

To achieve what you need

> set to 0, meaning that all
clauses are optional.

  you can combine the dismax with match all query *:*

On Thu, Jan 4, 2024 at 4:43 PM Anna Ruggero  wrote:

> Hi everyone,
>
> I have this query that uses the Boolean Query Parser:
> *http://localhost:8983/solr/books/select
> ?*
> *q={!bool filter=$**categoryIds** should=$queryTerms}&*
> *categoryIds**=**category:(1 2 5 7)**&*
> *queryTerms=**content**:(user text)*
>
> I would like to achieve the same result using the eDisMax query parser, but
> I encountered some problems with the minimum should match clause.
>
> Here is the query:
> *http://localhost:8983/solr/books/select
> ?*
> *fq=category:(1 2 5 7)&*
> *q=user text&*
> *df=content&*
> *defType=edismax*
>
> It returns only documents having "*user"* or "*text"* in the content field
> (but at least one of them is needed). If no documents contain those words
> in the *content* field, I have zero results returned.
> From the debug, I could see that the parsed query is:
> *+(DisjunctionMaxQuery((**content**:user)) DisjunctionMaxQuery((**content*
> *:text)))*
>
> By default the minimum should match parameter is set to 0, meaning that all
> clauses are optional. Why is there the + operator before the two
> disjunction queries? I wouldn't expect to see it. I would expect to see all
> documents returned even if none of them match any term in q.
>
> Also, if I explicitly set mm to 1 like:
> *http://localhost:8983/solr/books/select
> ?*
> *fq=category:(1 2 5 7)&*
> *q=user text&*
> *df=content&*
> *defType=edismax&*
> *mm=1*
>
> The parsed query is:
> *+(DisjunctionMaxQuery((**content**:**user**)) DisjunctionMaxQuery((*
> *content**:**text*
> *)))~1*
> Whose meaning is the same as the query before (with mm=0).
>
> What is the difference between mm=0 and mm=1?
>
>
> Thank you!
>
> *Anna Ruggero*
> R&D Machine Learning / Software Engineer, Search Consultant
> e-mail: a.rugg...@sease.io
>
>
> *Sease* - Information Retrieval Applied
> Consulting | Training | Open Source
> Sease.io 
> LinkedIn  | Twitter
>  | Youtube
>  | Github
> 
>


-- 
Sincerely yours
Mikhail Khludnev

Re: ExitableDirectoryReader warning

[David Smiley]

This is the ramification of using "timeAllowed" (a request param to a
search)

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Thu, Jan 4, 2024 at 2:18 PM rajani m  wrote:

> Hi,
>
> What scenarios would solr warn about "ExitableDirectoryReader" ? I see
> several "ExitableDirectoryReader$ExitingReaderException: The request took
> too long to iterate over terms.". What causes this this? Is it a shard size
> issue or lack of any resources, heap, ram or cpu?  .
>
> Log snippets -
>
> org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> The request took too long to iterate over terms. Timeout: timeoutAt:
> 2101774016440850 (System.nanoTime(): 2101774068035377),
>
> TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@79b2db56
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.checkTimeoutWithSampling(ExitableDirectoryReader.java:749)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTermsEnum.(ExitableDirectoryReader.java:737)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableTerms.iterator(ExitableDirectoryReader.java:719)
> at
> org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:116)
> at org.apache.lucene.index.TermStates.build(TermStates.java:102)
> at
> org.apache.lucene.search.PhraseQuery$1.getStats(PhraseQuery.java:461)
> at
> org.apache.lucene.search.PhraseWeight.(PhraseWeight.java:44)
> at
> org.apache.lucene.search.PhraseQuery$1.(PhraseQuery.java:441)
> at
> org.apache.lucene.search.PhraseQuery.createWeight(PhraseQuery.java:441)
> at
> org.apache.lucene.search.BoostQuery.createWeight(BoostQuery.java:124)
> at
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:869)
> at
> org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:59)
> at
> org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:237)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery.createWeight(FunctionScoreQuery.java:113)
>
>
> (from a different time window)
>
>
> org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException:
> The request took too long to iterate over doc values. Timeout:
> timeoutAt: 2103004456482731 (System.nanoTime(): 2103004484203805),
>
> DocValues=org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$2@2e1600c4
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader.checkAndThrow(ExitableDirectoryReader.java:394)
> at
> org.apache.lucene.index.ExitableDirectoryReader$ExitableFilterAtomicReader$1.advance(ExitableDirectoryReader.java:124)
> at
> org.apache.lucene.queries.function.valuesource.IntFieldSource$1.exists(IntFieldSource.java:81)
> at
> org.apache.lucene.queries.function.valuesource.MinFloatFunction.func(MinFloatFunction.java:39)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.valuesource.SumFloatFunction.func(SumFloatFunction.java:38)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.docvalues.FloatDocValues.doubleVal(FloatDocValues.java:66)
> at
> org.apache.solr.search.ValueSourceParser$33.func(ValueSourceParser.java:587)
> at
> org.apache.solr.search.ValueSourceParser$DoubleParser$Function$1.doubleVal(ValueSourceParser.java:1666)
> at
> org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:51)
> at
> org.apache.lucene.queries.function.valuesource.ProductFloatFunction.func(ProductFloatFunction.java:38)
> at
> org.apache.lucene.queries.function.valuesource.MultiFloatFunction$1.floatVal(MultiFloatFunction.java:83)
> at
> org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:126)
> at
> org.apache.lucene.search.DisjunctionSumScorer.score(DisjunctionSumScorer.java:41)
>