I have filed a Jira SOLR-17018 <https://issues.apache.org/jira/browse/SOLR-17018> explaining the issue and think it is just a certain type of queries causing the issue and not any bug, but still looking into it to see if anything else.
On Wed, Sep 27, 2023 at 11:38 AM rajani m <rajinima...@gmail.com> wrote: > Any alternative to timeout LTR re-ranker similar to the "timeAllowed" that > limits the main ranker? or is there a way to tell if a query is expensive > for ltr given a number of terms or has N hits? > > The LTR queries are taking forever and timing out at some point with > "ExitableDirectoryReader > '' exception, and at the same time this is causing the heap to completely > fill up and trigger an oom. An oom that invokes oom killer script that > brings solr down for sometime. Errors stack trace here on the gist > <https://gist.github.com/rajanim/1ee04de066aa4da6f0916f0c38274489> > > > 1. too long to iterate over terms. Timeout: timeoutAt: 946228141325435 > (System.nanoTime(): 946228142713792), > > TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@5f420d22 > 2. at > org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:163) > ~[?:?] > > > > 1. at > > org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:144) > ~[?:?] > 2. at > org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:107) > ~[?:?] > 3. at > > org.apache.solr.ltr.feature.SolrFeature$SolrFeatureWeight.scorer(SolrFeature.java:229) > ~[?:?] > 4. at > > org.apache.solr.ltr.LTRScoringQuery$ModelWeight.scorer(LTRScoringQuery.java:500) > ~[?:?] > > > On Tue, Sep 26, 2023 at 10:20 PM rajani m <rajinima...@gmail.com> wrote: > >> Hi Solr Users, >> >> Is there an alternative way to time limit an LTR query and request >> main ranker results if a reranker such as an LTR query takes more time than >> allowed? I see there is a SOLR-14607 >> <https://issues.apache.org/jira/browse/SOLR-14607> regarding LTR time >> allowed issue, but it is open. Wondering if there are any other >> alternatives to handle this? >> >> Some ltr based rerank queries are returning the following exception and >> causing solr heap to fill up before it is garbage collected by g1gc hence >> reporting a full heap usage and triggering oom.I am looking for solutions >> to prevent it from happening. Any suggestions appreciated. >> >> error - >> Caused by: >> org.apache.lucene.index.ExitableDirectoryReader$ExitingReaderException: The >> request took too long to iterate over point values. >> >
