Hi Solr Users and Devs,
Solr query with LTR as a re-ranker is using full heap all of sudden and
triggering STW pause. Could you please take a look and let me know your
thoughts? What is causing this? The STW is putting nodes in an unhealthy
state causing nodes to restart and bringing the entire cluster down.
As per logs, the issue seems to be related to LTR generating features at
query time. The model has 12 features and most features are solr query and
few field values. The error from the logs is copied below[2]. I'd say this
is a major bug as G1GC is supposed to avoid STW. What are your thoughts?
[1] q=((color) OR (colorable AND colourable) OR (colorant) OR (colorata) OR
(colorear) OR (colored) OR (colorete) OR (colorie) OR (coloring AND
colouring) OR (colors) OR (colour) OR (colourfulness) OR (colourations) OR
(coloured) OR (colourful) OR (colouring) OR (colourist AND colorist) OR
(complexion) OR (dyestuffs) OR (flush) OR (hue) OR (paint) OR (people) OR
(rosy) OR (stainable) OR (stained) OR (colored) OR (coloring) OR (tinted)
OR (turn) OR (colora))&defType=edismax&qf=keywords description title
body&rq={!ltr model=v1_20230302_model reRankDocs=1000 efi.q=$q}}
[2] org.apache.solr.common.SolrException: java.lang.RuntimeException:
Exception from createWeight for SolrFeature [name=keyword_pf3,
params={q={!edismax qf=keyword pf3=keyword^0.2}${q}}] The request took too
long to iterate over terms. Timeout: timeoutAt: 2027319049754068
(System.nanoTime(): 2027323098817134),
TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@1155724b
=> org.apache.solr.common.SolrException: java.lang.RuntimeException:
Exception from createWeight for SolrFeature [name=keyword_pf3,
params={q={!sdismax qf=keyword_pf3=keyword^0.2}${q}}] The request took too
long to iterate over terms. Timeout: timeoutAt: 2027319049754068
(System.nanoTime(): 2027323098817134),
TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@1155724b
at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:163)
org.apache.solr.common.SolrException: java.lang.RuntimeException: Exception
from createWeight for SolrFeature.
