[dev help wanted] upgrade Solr to use Lucene 9.9.1
Hi Everyone, Apache Lucene 9.9.1 was recently released - would you be interested in contributing to update Solr to use it? Please see https://issues.apache.org/jira/browse/SOLR-17097 for details. Thanks, Christine
Re: [dev help wanted] upgrade Solr to use Lucene 9.9.1
Hello Christine, I would be interested in it, but never done it before. So let me try and create a PR soon. On Wed, Jan 3, 2024 at 5:26 PM Christine Poerschke (BLOOMBERG/ LONDON) < cpoersc...@bloomberg.net> wrote: > Hi Everyone, > > Apache Lucene 9.9.1 was recently released - would you be interested in > contributing to update Solr to use it? > > Please see https://issues.apache.org/jira/browse/SOLR-17097 for details. > > Thanks, > Christine >
Solr query using full heap and triggers stop the world pause
Hi Solr Users and Devs,
Solr query with LTR as a re-ranker is using full heap all of sudden and
triggering STW pause. Could you please take a look and let me know your
thoughts? What is causing this? The STW is putting nodes in an unhealthy
state causing nodes to restart and bringing the entire cluster down.
As per logs, the issue seems to be related to LTR generating features at
query time. The model has 12 features and most features are solr query and
few field values. The error from the logs is copied below[2]. I'd say this
is a major bug as G1GC is supposed to avoid STW. What are your thoughts?
[1] q=((color) OR (colorable AND colourable) OR (colorant) OR (colorata) OR
(colorear) OR (colored) OR (colorete) OR (colorie) OR (coloring AND
colouring) OR (colors) OR (colour) OR (colourfulness) OR (colourations) OR
(coloured) OR (colourful) OR (colouring) OR (colourist AND colorist) OR
(complexion) OR (dyestuffs) OR (flush) OR (hue) OR (paint) OR (people) OR
(rosy) OR (stainable) OR (stained) OR (colored) OR (coloring) OR (tinted)
OR (turn) OR (colora))&defType=edismax&qf=keywords description title
body&rq={!ltr model=v1_20230302_model reRankDocs=1000 efi.q=$q}}
[2] org.apache.solr.common.SolrException: java.lang.RuntimeException:
Exception from createWeight for SolrFeature [name=keyword_pf3,
params={q={!edismax qf=keyword pf3=keyword^0.2}${q}}] The request took too
long to iterate over terms. Timeout: timeoutAt: 2027319049754068
(System.nanoTime(): 2027323098817134),
TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@1155724b
=> org.apache.solr.common.SolrException: java.lang.RuntimeException:
Exception from createWeight for SolrFeature [name=keyword_pf3,
params={q={!sdismax qf=keyword_pf3=keyword^0.2}${q}}] The request took too
long to iterate over terms. Timeout: timeoutAt: 2027319049754068
(System.nanoTime(): 2027323098817134),
TermsEnum=org.apache.lucene.codecs.lucene90.blocktree.SegmentTermsEnum@1155724b
at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:163)
org.apache.solr.common.SolrException: java.lang.RuntimeException: Exception
from createWeight for SolrFeature.
Restore from GCS stopped working after upgrading from solr 8.11.1 to 9.4.0
Hi Solr Users, We recently upgraded Solr from 8.11.1 to 9.4.0 after which the restore from GCS functionality stopped working. Solr uses the GCSBackupRepository plugin to backup and restore Lucene segments. Prior to the upgrade both backup and restore operations were happening correctly. But post upgrade the backup happens to GCS but when we try to restore from the backup in GCS most of the shards get stuck after copying some files and after some time the solr container crashes. Interestingly the restore gets stuck after copying 129GB of files from GCS every time. Eventually solr logs show the following exception: c.g.a.c.h.HttpTransport exception thrown while executing request java.net.UnknownHostException: oauth2.googleapis.com Solr 8 and Solr 9 restore code doesn't seem to change much. However, it looks that solr 8 was using google.cloud.storage 1.x and solr 9 moved to 2.x version of that library. Has anyone else run into this issue with solr backup and restore in GCP? Thanks, -- Shubhro Roy, Engineering Manager, Search & AI 900 Jefferson Ave Redwood City, CA 94063
