Re: OutofMemory Error in solr 6.5

[Satya Nand]

Hi Dominique,

You don't provide information about the number of documents. Anyway, all
> your cache size and mostly initial size are big. Cache are stored in JVM
> heap.

Document count is 101893353.

About cache size, most is not always better. Did you make some performance
> benchmarks in order to set these values ?

We increased cache size in the hope to reduce some response time, We
heavily use group queries with 7-8 boost factors. The average response time
on this document set is 136 ms. We receive approx 65 requests/second in
peak hours. The replication interval is 3 hours.

The most strange thing about is that system keeps running for days without
any issue, So I believe cache size should not be an issue. If the cache
size had been the culprit, the issue would have been frequent.  isn't it?



On Mon, Aug 9, 2021 at 6:44 PM Dominique Bejean 
wrote:

> Hi,
>
> You don't provide information about the number of documents. Anyway, all
> your cache size and mostly initial size are big. Cache are stored in JVM
> heap.
>
> About cache size, most is not always better. Did you make some performance
> benchmarks in order to set these values ?
>
> Try with the default values, after a few hours check cumulative caches
> statistics in order to decide if you need to increase their sizes. The
> objective is not to have cumulative_hitratio to 100%. There isn't ideal
> value as it is really related to your datas, to the user's queries, to how
> you build your queries ... but 70% is a good value. At some point
> increasing the size again and again won't increase cumulative_hitratio a
> lot as it is a logarithmic curve.
>
> Check also the heap usage also with your JVM GC logs and a tool like
> gceasy.io
>
> Regards
>
> Dominique
>
>
>
>
> Le lun. 9 août 2021 à 07:44, Satya Nand 
> a écrit :
>
> > Hi,
> > We are facing a strange issue in our solr system. Most of the days it
> keeps
> > running fine but once or twice in a month, we face OutofMemory on solr
> > servers.
> >
> > We are using Leader-Follower architecture, one Leader and 4 followers.
> > Strangely we get OutofMemory error on all follower servers.
> > Before the OutOfMemory this exception is found on all servers.
> >
> > Aug, 04 2021 15:26:11 org.apache.solr.servlet.HttpSolrCall
> > search-central-prd-solr-temp1
> > ERROR: null:java.lang.NullPointerException search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:617)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:240)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:2027)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1844)
> > search-central-prd-solr-temp1
> > at
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:547)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
> > search-central-prd-solr-temp1
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
> > search-central-prd-solr-temp1
> > at
> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
> > search-central-prd-solr-temp1
> > at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
> > search-central-prd-solr-temp1
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> > search-central-prd-solr-temp1
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> > search-central-prd-solr-temp1
> > at
> >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> > search-central-prd-solr-temp1

Re: OutofMemory Error in solr 6.5

[Satya Nand]

Hi Shawn,
>
>
> Do you have the actual OutOfMemoryError exception?  Can we see that?
> There are several resources other than heap memory that will result in
> OOME if they are exhausted.  It's important to be investigating the
> correct resource.

*Exception:*
 Aug, 04 2021 15:38:36 org.apache.solr.servlet.HttpSolrCall
search-central-prd-solr-temp1
ERROR: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
heap space search-central-prd-solr-temp1
at
org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:676)
search-central-prd-solr-temp1
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:544)
search-central-prd-solr-temp1
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
search-central-prd-solr-temp1
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
search-central-prd-solr-temp1
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
search-central-prd-solr-temp1
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
search-central-prd-solr-temp1
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
search-central-prd-solr-temp1
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
search-central-prd-solr-temp1
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
search-central-prd-solr-temp1
at org.eclipse.jetty.server.Server.handle(Server.java:534)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
search-central-prd-solr-temp1
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
search-central-prd-solr-temp1
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
search-central-prd-solr-temp1
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
search-central-prd-solr-temp1
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
search-central-prd-solr-temp1
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
search-central-prd-solr-temp1
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
search-central-prd-solr-temp1
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
search-central-prd-solr-temp1
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
search-central-prd-solr-temp1
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
search-central-prd-solr-temp1
 at java.lang.Thread.run(Thread.java:748)
search-central-prd-solr-temp1
Caused by: java.lang.OutOfMemoryError: Java heap space
search-central-prd-solr-temp1
at org.apache.lucene.util.FixedBitSet.clone(FixedBitSet.java:480)
search-central-prd-solr-temp1
at
org.apache.solr.search.DocSetBase.intersection(DocSetBase.java:124)
search-central-prd-solr-temp1
at org.apache.solr.search.BitDocSet.intersection(BitDocSet.java:40)
search-central-prd-solr-temp1
at
org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1482)
search-central-prd-solr-temp1
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1968)
search-central-prd-solr-temp1
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1844)
search-central-prd-solr-temp1
 

Heatmap facets "Too many cells" IllegalArgumentException

[nicolas n]

Hello everyone,

We are currently using Solr 8.7 facets search to render a server generated 
heatmap.
Our issue is the following :

Solr often return this kind of exception => 
"java.lang.IllegalArgumentException: Too many cells (743 x 261) for level 5 
shape 
Rect(minX=-18.610839843750004,maxX=13.996582031250002,minY=40.53050177574321,maxY=51.97134580885172)"

Here the facet json query was => 
{
  "coord": {
  "type": "heatmap",
  "field": "coord_fieldname",
  "geom": "[\"-18.610839843750004 40.53050177574321\" TO \"13.996582031250002 
51.97134580885172\"]",
  "distErrPct":0.025
   }
}

The same query works if we put the distErrPct at 0.03.

We found that these lines of code, in 
org.apache.lucene.spatial.prefix.HeatmapFacetCounter class were generating this 
exception:

  45   public static final int MAX_ROWS_OR_COLUMNS = (int) 
Math.sqrt(ArrayUtil.MAX_ARRAY_LENGTH);
  ..
 140 if (columns > MAX_ROWS_OR_COLUMNS || rows > MAX_ROWS_OR_COLUMNS || columns 
* rows > maxCells) {
 141   throw new IllegalArgumentException(
 142   "Too many cells ("
 143   + columns
 144   + " x "
 145   + rows
 146   + ") for level "
 147   + facetLevel
 148   + " shape "
 149   + inputRect);
 150 }

Althought we don"t really understand how these maximum rows and columns counts 
are calculated.
We would like to obtain maximum precision when calculating our heatmap.
Should we adapt distErrPct before each query in order to avoid this kind of 
response from Solr ? (if yes, how so ?)
Or is there something else we are missing ?
Because Solr succeed in processing this query for most of our "geom" rectangles 
in parameter, but it fails for some of them. The rectangles we send always have 
same proportions, just the "zoom" level differs.

Thanks for your answers !

Solr Operator vs Bitnami helm chart

[Tomer]

Hello,
I'm looking for the best way to go forward for a production-ready Solr
deployment
I've noticed there's 2 prevalent options:
1. Solr Operator
2. Bitnami helm chart

Any existing resources I can use to make a decision?
Thanks

Solr russification

[Егор Иванов]

Hello, my name is Egor
Is there any way to translate Solr's GUI to russian?
Looking forward to hearing from you

Re: Solr russification

[Eric Pugh]

Nothing built in today.   The Solr GUI is written in AngularJS, and while it 
references support for i18n, no one has taken that step.  
https://docs.angularjs.org/guide/i18n#how-does-angularjs-support-i18n-l10n-

This type of feature is why we’d love to see someone step up and move the GUI 
to more modern platform that more easily supports this kind of feature.

> On Aug 10, 2021, at 6:36 AM, Егор Иванов  wrote:
> 
> Hello, my name is Egor
> Is there any way to translate Solr's GUI to russian?
> Looking forward to hearing from you

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Field value feature returning only 0s - LTR feature engineering

[Nishanth Nayakanti]

Hello everyone,

We are using Solr 8.5 and facing the following issue.

We want to use number of clicks of a documents/page as a feature in LTR model. 
We are using in-place updates to set/upload the number of clicks for each 
document.  When I try to extract features from Solr, all the documents/rows 
have values as 0 for clicks feature.

Sample feature
{
  "name":"pmetrics_its_nid-ga_metric1-sum",
  "class":"org.apache.solr.ltr.feature.FieldValueFeature",
  "params":{"field":"pmetrics_its_nid-ga_metric1-sum"},
  "store":"_DEFAULT_"
}

Solr Config




Am I missing something here ?

Thanks for your answers
Nishanth.
Disclaimer: This message and any attachments are solely intended for the 
addressee(s). It may also be TA DIGITAL confidential, privileged and / or 
subject to copyright. Access to this email by anyone else is unauthorized. If 
you are not the intended recipient, any disclosure, copying, distribution or 
any action taken or omitted to be taken in reliance on it, is prohibited and 
may be unlawful. If you have received this in error, please notify the sender 
immediately by return e-mail and delete it from your computer. While all care 
has been taken, TA DIGITAL management disclaims all liabilities for loss or 
damages to person(s) or properties arising from misuse of any information 
provided or the message being infected by computer virus or other contamination.

Re: Solr russification

[Dave]

Cant chrome translate the page or is there too much JavaScript?

> On Aug 10, 2021, at 8:44 AM, Eric Pugh  
> wrote:
> 
> Nothing built in today.   The Solr GUI is written in AngularJS, and while it 
> references support for i18n, no one has taken that step.  
> https://docs.angularjs.org/guide/i18n#how-does-angularjs-support-i18n-l10n-
> 
> This type of feature is why we’d love to see someone step up and move the GUI 
> to more modern platform that more easily supports this kind of feature.
> 
>> On Aug 10, 2021, at 6:36 AM, Егор Иванов  wrote:
>> 
>> Hello, my name is Egor
>> Is there any way to translate Solr's GUI to russian?
>> Looking forward to hearing from you
> 
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com  
> | My Free/Busy   
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> 
> 
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
> 

Re: Field value feature returning only 0s - LTR feature engineering

[Dave]

I’m confused. You don’t store it, nor index it?

> On Aug 10, 2021, at 9:06 AM, Nishanth Nayakanti 
>  wrote:
> 
> Hello everyone,
> 
> We are using Solr 8.5 and facing the following issue.
> 
> We want to use number of clicks of a documents/page as a feature in LTR 
> model. We are using in-place updates to set/upload the number of clicks for 
> each document.  When I try to extract features from Solr, all the 
> documents/rows have values as 0 for clicks feature.
> 
> Sample feature
> {
>  "name":"pmetrics_its_nid-ga_metric1-sum",
>  "class":"org.apache.solr.ltr.feature.FieldValueFeature",
>  "params":{"field":"pmetrics_its_nid-ga_metric1-sum"},
>  "store":"_DEFAULT_"
> }
> 
> Solr Config
> 
>  docValues="true"/>
> 
> 
> Am I missing something here ?
> 
> Thanks for your answers
> Nishanth.
> Disclaimer: This message and any attachments are solely intended for the 
> addressee(s). It may also be TA DIGITAL confidential, privileged and / or 
> subject to copyright. Access to this email by anyone else is unauthorized. If 
> you are not the intended recipient, any disclosure, copying, distribution or 
> any action taken or omitted to be taken in reliance on it, is prohibited and 
> may be unlawful. If you have received this in error, please notify the sender 
> immediately by return e-mail and delete it from your computer. While all care 
> has been taken, TA DIGITAL management disclaims all liabilities for loss or 
> damages to person(s) or properties arising from misuse of any information 
> provided or the message being infected by computer virus or other 
> contamination.

Re: Solr russification

[Stephen Boesch]

automated translations are rough at best

On Tue, 10 Aug 2021 at 06:16, Dave  wrote:

> Cant chrome translate the page or is there too much JavaScript?
>
> > On Aug 10, 2021, at 8:44 AM, Eric Pugh 
> wrote:
> >
> > Nothing built in today.   The Solr GUI is written in AngularJS, and
> while it references support for i18n, no one has taken that step.
> https://docs.angularjs.org/guide/i18n#how-does-angularjs-support-i18n-l10n-
> >
> > This type of feature is why we’d love to see someone step up and move
> the GUI to more modern platform that more easily supports this kind of
> feature.
> >
> >> On Aug 10, 2021, at 6:36 AM, Егор Иванов  wrote:
> >>
> >> Hello, my name is Egor
> >> Is there any way to translate Solr's GUI to russian?
> >> Looking forward to hearing from you
> >
> > ___
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>

Re: Solr russification

[Dave]

Gotcha, I don’t know Russian so I didn’t know that. the admin ui has very 
limited English to translate there aren’t paragraphs, unless you want the 
results translated, otherwise it’s pretty straight forward I would have thought 

> On Aug 10, 2021, at 9:19 AM, Stephen Boesch  wrote:
> 
> automated translations are rough at best
> 
>> On Tue, 10 Aug 2021 at 06:16, Dave  wrote:
>> 
>> Cant chrome translate the page or is there too much JavaScript?
>> 
>>> On Aug 10, 2021, at 8:44 AM, Eric Pugh 
>> wrote:
>>> 
>>> Nothing built in today.   The Solr GUI is written in AngularJS, and
>> while it references support for i18n, no one has taken that step.
>> https://docs.angularjs.org/guide/i18n#how-does-angularjs-support-i18n-l10n-
>>> 
>>> This type of feature is why we’d love to see someone step up and move
>> the GUI to more modern platform that more easily supports this kind of
>> feature.
>>> 
 On Aug 10, 2021, at 6:36 AM, Егор Иванов  wrote:
 
 Hello, my name is Egor
 Is there any way to translate Solr's GUI to russian?
 Looking forward to hearing from you
>>> 
>>> ___
>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
>> http://www.opensourceconnections.com <
>> http://www.opensourceconnections.com/> | My Free/Busy <
>> http://tinyurl.com/eric-cal>
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> 
>>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless of
>> whether attachments are marked as such.
>>> 
>> 

Re: Solr russification

[Thomas Corthals]

Machine translation often does a better job with paragraphs because the
words are in context. Single word labels such as "Logging" are what gets it
confused because it can't always figure out if that's about log files or
felling trees.

Op di 10 aug. 2021 om 15:27 schreef Dave :

> Gotcha, I don’t know Russian so I didn’t know that. the admin ui has very
> limited English to translate there aren’t paragraphs, unless you want the
> results translated, otherwise it’s pretty straight forward I would have
> thought
>
> > On Aug 10, 2021, at 9:19 AM, Stephen Boesch  wrote:
> >
> > automated translations are rough at best
> >
> >> On Tue, 10 Aug 2021 at 06:16, Dave 
> wrote:
> >>
> >> Cant chrome translate the page or is there too much JavaScript?
> >>
> >>> On Aug 10, 2021, at 8:44 AM, Eric Pugh <
> ep...@opensourceconnections.com>
> >> wrote:
> >>>
> >>> Nothing built in today.   The Solr GUI is written in AngularJS, and
> >> while it references support for i18n, no one has taken that step.
> >>
> https://docs.angularjs.org/guide/i18n#how-does-angularjs-support-i18n-l10n-
> >>>
> >>> This type of feature is why we’d love to see someone step up and move
> >> the GUI to more modern platform that more easily supports this kind of
> >> feature.
> >>>
>  On Aug 10, 2021, at 6:36 AM, Егор Иванов 
> wrote:
> 
>  Hello, my name is Egor
>  Is there any way to translate Solr's GUI to russian?
>  Looking forward to hearing from you
> >>>
> >>> ___
> >>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> |
> >> http://www.opensourceconnections.com <
> >> http://www.opensourceconnections.com/> | My Free/Busy <
> >> http://tinyurl.com/eric-cal>
> >>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >>
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >
> >>
> >>> This e-mail and all contents, including attachments, is considered to
> be
> >> Company Confidential unless explicitly stated otherwise, regardless of
> >> whether attachments are marked as such.
> >>>
> >>
>

RE: Field value feature returning only 0s - LTR feature engineering

[Nishanth Nayakanti]

Hi Dave,

Thanks for responding

I think that config setting is a pre-requisite for inplace updates as defined 
here 
https://solr.apache.org/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-Example.1

Here is from Solr documentation.

An atomic update operation is performed using this approach only when the 
fields to be updated meet these three conditions:

are non-indexed (indexed="false"), non-stored (stored="false"), single valued 
(multiValued="false") numeric docValues (docValues="true") fields;

the _version_ field is also a non-indexed, non-stored single valued docValues 
field; and,

copy targets of updated fields, if any, are also non-indexed, non-stored single 
valued numeric docValues fields.

Thanks
Nishanth

-Original Message-
From: Dave 
Sent: Tuesday, August 10, 2021 6:47 PM
To: users@solr.apache.org
Cc: Madhuluck Kumar 
Subject: Re: Field value feature returning only 0s - LTR feature engineering

***Caution: External Email***

This email originated from an external sender. Do not click on links or open 
attachments unless you were expecting this message from this sender.

I’m confused. You don’t store it, nor index it?

> On Aug 10, 2021, at 9:06 AM, Nishanth Nayakanti 
>  wrote:
>
> Hello everyone,
>
> We are using Solr 8.5 and facing the following issue.
>
> We want to use number of clicks of a documents/page as a feature in LTR 
> model. We are using in-place updates to set/upload the number of clicks for 
> each document.  When I try to extract features from Solr, all the 
> documents/rows have values as 0 for clicks feature.
>
> Sample feature
> {
>  "name":"pmetrics_its_nid-ga_metric1-sum",
>  "class":"org.apache.solr.ltr.feature.FieldValueFeature",
>  "params":{"field":"pmetrics_its_nid-ga_metric1-sum"},
>  "store":"_DEFAULT_"
> }
>
> Solr Config
>
>  docValues="true"/>
>
>
> Am I missing something here ?
>
> Thanks for your answers
> Nishanth.
> Disclaimer: This message and any attachments are solely intended for the 
> addressee(s). It may also be TA DIGITAL confidential, privileged and / or 
> subject to copyright. Access to this email by anyone else is unauthorized. If 
> you are not the intended recipient, any disclosure, copying, distribution or 
> any action taken or omitted to be taken in reliance on it, is prohibited and 
> may be unlawful. If you have received this in error, please notify the sender 
> immediately by return e-mail and delete it from your computer. While all care 
> has been taken, TA DIGITAL management disclaims all liabilities for loss or 
> damages to person(s) or properties arising from misuse of any information 
> provided or the message being infected by computer virus or other 
> contamination.
Disclaimer: This message and any attachments are solely intended for the 
addressee(s). It may also be TA DIGITAL confidential, privileged and / or 
subject to copyright. Access to this email by anyone else is unauthorized. If 
you are not the intended recipient, any disclosure, copying, distribution or 
any action taken or omitted to be taken in reliance on it, is prohibited and 
may be unlawful. If you have received this in error, please notify the sender 
immediately by return e-mail and delete it from your computer. While all care 
has been taken, TA DIGITAL management disclaims all liabilities for loss or 
damages to person(s) or properties arising from misuse of any information 
provided or the message being infected by computer virus or other contamination.

Re: Solr russification

[Mark H. Wood]

On Tue, Aug 10, 2021 at 04:08:04PM +0200, Thomas Corthals wrote:
> Machine translation often does a better job with paragraphs because the
> words are in context. Single word labels such as "Logging" are what gets it
> confused because it can't always figure out if that's about log files or
> felling trees.
> 
> Op di 10 aug. 2021 om 15:27 schreef Dave :
> 
> > Gotcha, I don’t know Russian so I didn’t know that. the admin ui has very
> > limited English to translate there aren’t paragraphs, unless you want the
> > results translated, otherwise it’s pretty straight forward I would have
> > thought

That makes sense, but the first issue is going to be:  was the code
written to be localizable?  It sounds to me, from a previous post,
as if the answer is "no", though the UI kit seems to support it.

To make code localizable, you first have to internationalize it.  That
means moving every scrap of user-visible text to a message catalog.
Every button label; every error message; many link bodies; every bit of
explanation.  Otherwise, every translator will have to hunt down all
of these places, and that takes a skill set rather different from
those involved in translation -- people who are good at both are not
common.

Once you have all the text in a catalog, translation mostly involves
reading one catalog and writing another.  Upgrading starts with
listing the changes in one catalog and making corresponding changes in
another.  That is much easier than trawling through code looking for
things that might have changed.  Machine translation *may* help.

-- 
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu


signature.asc
Description: PGP signature

How to solve: ERROR Appender solr-async is unable to write primary appenders. queue is full

[Howard Edward]

Hello,

Started receiving this message. How to solve this problem?

2021-08-09 11:22:49,871 qtp399573350-29619 ERROR Appender solr-async is
unable to write primary appenders. queue is full

2021-08-09 11:22:49,889 qtp399573350-29619 ERROR Appender solr-async is
unable to write primary appenders. queue is full

2021-08-09 11:22:49,891 qtp399573350-29461 ERROR Appender solr-async is
unable to write primary appenders. queue is full

2021-08-09 11:22:49,891 qtp399573350-29461 ERROR Appender solr-async is
unable to write primary appenders. queue is full

2021-08-09 11:34:30,798 qtp399573350-29606 ERROR Appender solr-async is
unable to write primary appenders. queue is full

2021-08-09 11:37:41.387:INFO:oejs.ServerConnector:ShutdownMonitor: Stopped
ServerConnector@2dfbde96{HTTP/1.1,[http/1.1]}{0.0.0.0:8765}

2021-08-09 11:37:43.599:WARN:oejs.ServletHandler:qtp399573350-29564:

javax.servlet.ServletException: A MultiException has 1 exceptions.  They
are:|1. java.lang.IllegalStateException:
ServiceLocatorImpl(GuiceServiceLocator-1,3,1623002407) has been shut down|

at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:421)


Thanx!

Ken

Re: Solr russification

[dmitri maziuk]

On 2021-08-10 8:18 AM, Stephen Boesch wrote:

automated translations are rough at best



As a native Russian speaker I promise you that non-automated ones are 
just as bad at best.


Dima

Re: Solr Operator vs Bitnami helm chart

[Houston Putman]

Hey Tomer,

I'm not aware of any resources that compare the two, however the Solr
Operator is the official project-supported way of running Solr on
Kubernetes.
You can find more information on the project at:
https://solr.apache.org/operator

Note, the next version of the Solr Operator (v0.4.0) will include a helm
chart to deploy Solr, much like the Bitnami Helm Chart, but you will have
to deploy the Solr Operator as well.

- Houston

On Tue, Aug 10, 2021 at 7:59 AM Tomer  wrote:

> Hello,
> I'm looking for the best way to go forward for a production-ready Solr
> deployment
> I've noticed there's 2 prevalent options:
> 1. Solr Operator
> 2. Bitnami helm chart
>
> Any existing resources I can use to make a decision?
> Thanks
>

Re: OutofMemory Error in solr 6.5

[Dominique Bejean]

Pretty sure the issue is caused by caches size at new searcher warmup time.

Dominique


Le mar. 10 août 2021 à 09:07, Satya Nand  a
écrit :

> Hi Dominique,
>
> You don't provide information about the number of documents. Anyway, all
>> your cache size and mostly initial size are big. Cache are stored in JVM
>> heap.
>
> Document count is 101893353.
>
> About cache size, most is not always better. Did you make some performance
>> benchmarks in order to set these values ?
>
> We increased cache size in the hope to reduce some response time, We
> heavily use group queries with 7-8 boost factors. The average response time
> on this document set is 136 ms. We receive approx 65 requests/second in
> peak hours. The replication interval is 3 hours.
>
> The most strange thing about is that system keeps running for days without
> any issue, So I believe cache size should not be an issue. If the cache
> size had been the culprit, the issue would have been frequent.  isn't it?
>
>
>
> On Mon, Aug 9, 2021 at 6:44 PM Dominique Bejean 
> wrote:
>
>> Hi,
>>
>> You don't provide information about the number of documents. Anyway, all
>> your cache size and mostly initial size are big. Cache are stored in JVM
>> heap.
>>
>> About cache size, most is not always better. Did you make some performance
>> benchmarks in order to set these values ?
>>
>> Try with the default values, after a few hours check cumulative caches
>> statistics in order to decide if you need to increase their sizes. The
>> objective is not to have cumulative_hitratio to 100%. There isn't ideal
>> value as it is really related to your datas, to the user's queries, to how
>> you build your queries ... but 70% is a good value. At some point
>> increasing the size again and again won't increase cumulative_hitratio a
>> lot as it is a logarithmic curve.
>>
>> Check also the heap usage also with your JVM GC logs and a tool like
>> gceasy.io
>>
>> Regards
>>
>> Dominique
>>
>>
>>
>>
>> Le lun. 9 août 2021 à 07:44, Satya Nand > .invalid>
>> a écrit :
>>
>> > Hi,
>> > We are facing a strange issue in our solr system. Most of the days it
>> keeps
>> > running fine but once or twice in a month, we face OutofMemory on solr
>> > servers.
>> >
>> > We are using Leader-Follower architecture, one Leader and 4 followers.
>> > Strangely we get OutofMemory error on all follower servers.
>> > Before the OutOfMemory this exception is found on all servers.
>> >
>> > Aug, 04 2021 15:26:11 org.apache.solr.servlet.HttpSolrCall
>> > search-central-prd-solr-temp1
>> > ERROR: null:java.lang.NullPointerException search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:617)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:240)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:2027)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1844)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:547)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
>> > search-central-prd-solr-temp1
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
>> > search-central-prd-solr-temp1
>> > at
>> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
>> > search-central-prd-solr-temp1
>> > at
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> >
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>> > search-central-prd-solr-temp1
>> > at
>> >
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>> > sear

Re: Field value feature returning only 0s - LTR feature engineering

[Alessandro Benedetti]

Looking at the code of the FieldValueFeature in the LTR integration it
seems to be fully compatible with "DocValues" only fields :
org/apache/solr/ltr/feature/FieldValueFeature.java:128

It would require some additional investigation to understand why you get
only zeroes (have you tried with a different fieldType for example?)
--
Alessandro Benedetti
Apache Lucene/Solr Committer
Director, R&D Software Engineer, Search Consultant

www.sease.io


On Tue, 10 Aug 2021 at 16:25, Nishanth Nayakanti
 wrote:

> Hi Dave,
>
> Thanks for responding
>
> I think that config setting is a pre-requisite for inplace updates as
> defined here
> https://solr.apache.org/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-Example.1
>
> Here is from Solr documentation.
>
> An atomic update operation is performed using this approach only when the
> fields to be updated meet these three conditions:
>
> are non-indexed (indexed="false"), non-stored (stored="false"), single
> valued (multiValued="false") numeric docValues (docValues="true") fields;
>
> the _version_ field is also a non-indexed, non-stored single valued
> docValues field; and,
>
> copy targets of updated fields, if any, are also non-indexed, non-stored
> single valued numeric docValues fields.
>
> Thanks
> Nishanth
>
> -Original Message-
> From: Dave 
> Sent: Tuesday, August 10, 2021 6:47 PM
> To: users@solr.apache.org
> Cc: Madhuluck Kumar 
> Subject: Re: Field value feature returning only 0s - LTR feature
> engineering
>
> ***Caution: External Email***
>
> This email originated from an external sender. Do not click on links or
> open attachments unless you were expecting this message from this sender.
>
> I’m confused. You don’t store it, nor index it?
>
> > On Aug 10, 2021, at 9:06 AM, Nishanth Nayakanti <
> nishant...@tadigital.com.invalid> wrote:
> >
> > Hello everyone,
> >
> > We are using Solr 8.5 and facing the following issue.
> >
> > We want to use number of clicks of a documents/page as a feature in LTR
> model. We are using in-place updates to set/upload the number of clicks for
> each document.  When I try to extract features from Solr, all the
> documents/rows have values as 0 for clicks feature.
> >
> > Sample feature
> > {
> >  "name":"pmetrics_its_nid-ga_metric1-sum",
> >  "class":"org.apache.solr.ltr.feature.FieldValueFeature",
> >  "params":{"field":"pmetrics_its_nid-ga_metric1-sum"},
> >  "store":"_DEFAULT_"
> > }
> >
> > Solr Config
> >
> >  stored="false" docValues="true"/>
> >
> >
> > Am I missing something here ?
> >
> > Thanks for your answers
> > Nishanth.
> > Disclaimer: This message and any attachments are solely intended for the
> addressee(s). It may also be TA DIGITAL confidential, privileged and / or
> subject to copyright. Access to this email by anyone else is unauthorized.
> If you are not the intended recipient, any disclosure, copying,
> distribution or any action taken or omitted to be taken in reliance on it,
> is prohibited and may be unlawful. If you have received this in error,
> please notify the sender immediately by return e-mail and delete it from
> your computer. While all care has been taken, TA DIGITAL management
> disclaims all liabilities for loss or damages to person(s) or properties
> arising from misuse of any information provided or the message being
> infected by computer virus or other contamination.
> Disclaimer: This message and any attachments are solely intended for the
> addressee(s). It may also be TA DIGITAL confidential, privileged and / or
> subject to copyright. Access to this email by anyone else is unauthorized.
> If you are not the intended recipient, any disclosure, copying,
> distribution or any action taken or omitted to be taken in reliance on it,
> is prohibited and may be unlawful. If you have received this in error,
> please notify the sender immediately by return e-mail and delete it from
> your computer. While all care has been taken, TA DIGITAL management
> disclaims all liabilities for loss or damages to person(s) or properties
> arising from misuse of any information provided or the message being
> infected by computer virus or other contamination.
>

Re: help us!!!

[Alessandro Benedetti]

Hi Agree with Gora,
it could be anything, it requires some further investigation for sure, I
recommend to go for a consultancy service unless you are able to provide
more details here.

Regards
--
Alessandro Benedetti
Apache Lucene/Solr Committer
Director, R&D Software Engineer, Search Consultant

www.sease.io


On Fri, 6 Aug 2021 at 11:02, Gora Mohanty  wrote:

> On Fri, 6 Aug 2021 at 10:52, 839606848  wrote:
>
> > Hi Gora,
> > I'm glad to receive your reply.
> > Because of the confidentiality agreement, I can't provide some direct
> > information on this issue. I can give an example.
> >
>
> It becomes difficult in that case to provide any kind of support as the
> problem could be anywhere. Maybe you can hire a local consultant under a
> non-disclosure agreement.
>
> From your description, one thing that you might be missing is to do a
> commit after indexing new documents, and after deletion.
>
> Regards,
> Gora
>

Re: OutofMemory Error in solr 6.5

[Shawn Heisey]

On 8/10/2021 1:06 AM, Satya Nand wrote:

Document count is 101893353.


The OOME exception confirms that we are dealing with heap memory.  That 
means we won't have to look into the other resource types that can cause 
OOME.


With that document count, each filterCache entry is 12736670 bytes, plus 
some small number of bytes for java object overhead.  That's 12.7 
million bytes.


If your configured 4000 entry filterCache were to actually fill up, it 
would require nearly 51 billion bytes, and that's just for the one core 
with 101 million documents.  This is much larger than the 30GB heap you 
have specified ... I am betting that the filterCache is the reason 
you're hitting OOME.


You need to dramatically reduce the size of your filterCache.  Start 
with 256 and see what that gets you.  Solr ships with a size of 512. 
Also, see what you can do about making it so that there is a lot of 
re-use possible with queries that you put in the fq parameter.  It's 
better to have several fq parameters rather than one parameter with a 
lot of AND clauses -- much more chance of filter re-use.


I notice that you have autowarmCount set to 100 on two caches.  (The 
autowarmCount on the documentCache, which you have set to 512, won't be 
used -- that cache cannot be warmed directly.  It is indirectly warmed 
when the other caches are warmed.)  This means that every time you issue 
a commit that opens a new searcher, Solr will execute up to 200 queries 
as part of the cache warming.  This can make the warming take a VERY 
long time.  Consider reducing autowarmCount.  It's not causing your OOME 
problems, but it might be making commits take a very long time.


Thanks,
Shawn

Re: Time Routed Alias

[Matt Kuiper]

I found some helpful information while testing TRAs:

For our use-case I am hesitant to set up an autoDeleteAge (unless it can be
modified - still need to test).  So I wondered about a little more manual
delete management approach.

I confirmed that I cannot simply delete a collection that is registered as
part of a TRA.  The delete collection api call will fail with a message
that the collection is a part of the alias.

I did learn that I could use the same create TRA api call I used to create
the TRA, but modify the router.start to date more recent than one or more
of the older collections associated with the TRA. Then when I queried the
TRA, I only received documents from the collections after the new
router.start date. Also, I was now able to successfully delete the older
collections with a standard collection delete command.

I think this satisfies my initial use-case requirements to be able to
modify an existing TRA and delete older collections.

Matt

On Mon, Aug 9, 2021 at 11:27 AM Matt Kuiper  wrote:

> Hi Gus, Jan,
>
> I am considering implementing TRA for a large-scale Solr deployment.  Your
> Q&A is helpful!
>
> I am curious if you have experience/ideas regarding modifying the TR Alias
> when one desires to manually delete old collections or modify the
> router.autoDeleteAge to shorten or extend the delete age.  Here's a few
> specific questions?
>
> 1) Can you manually delete an old collection (via collection api) and then
> edit the start date (to a more recent date) of the TRA so that it no longer
> sees/processes the deleted collection?
> 2) Is the only way to manage the deletion of collections within a TRA
> using the automatic deletion configuration? The router.autoDeleteAge
> parameter.
> 3) If you can only manage deletes using the router.autoDeleteAge
> parameter, are you able to update this parameter to either:
>
>- Set the delete age earlier so that older collections are triggered
>for automatic deletion sooner?
>- Set the delete age to a larger value to extend the life of a
>collection?  Say you originally  would like the collections to stay around
>for 5 years, but then change your mind to 7 years.
>
> I will likely do some experimentation, but am interested to learn if you
> have covered these use-cases with TRA.
>
> Thanks,
> Matt
>
>
> On Fri, Aug 6, 2021 at 8:08 AM Gus Heck  wrote:
>
>> Hi Jan,
>>
>> The key thing to remember about TRA's (or any Routed Alias) is that it
>> only
>> actively does two things:
>> 1) Routes document updates to the correct collection by inspecting the
>> routed field in the document
>> 2) Detects when a new collection is required and creates it.
>>
>> If you don't send it data *nothing* happens. The collections are not
>> created until data requires them (with an async create possible when it
>> sees an update that has a timestamp "near" the next interval, see docs for
>> router.preemptiveCreateMath )
>>
>> A) Dave's half of our talk at 2018 activate talks about it:
>> https://youtu.be/RB1-7Y5NQeI?t=839
>> B) Time Routed Aliases are a means by which to automate creation of
>> collections and route documents to the created collections. Sizing, and
>> performance of the individual collections is not otherwise special, and
>> you
>> can interact with the collections individually after they are created,
>> with
>> the obvious caveats that you probably don't want to be doing things that
>> get them out of sync schema wise unless your client programs know how to
>> handle documents of both types etc. A less obvious consequence of the
>> routing is that your data must not ever republish the same document with a
>> different route key (date for TRA), since that can lead to duplicate id's
>> across collections. The "normal" use case is event data, things that
>> happened and are done, and are correctly recorded (or at least their time
>> is correctly recorded) the first time
>> C) Configure the higher number of replicas, remove old ones manually if
>> not
>> needed. At query time it's "just an alias". Managing collections based on
>> recency could be automated here, before autoscaling was deprecated I was
>> thinking that adding a couple of hooks into autoscaling such that it could
>> react to collection creation by a TRA specifically would get us to a place
>> much like Elastic's Hot/Warm architecture. I haven't kept track of what's
>> being done to replace auto scaling however. I think Atri was interested in
>> that at one point as well.
>> D) TRA's create collections under the hood with a CREATE command just like
>> you would manually (based on the config in the TRA). Anything in Solr that
>> would influence that placement should apply.
>> E) See D above, for fill rate, Utilizing new nodes over time should be as
>> simple as adding new nodes and waiting for new collections to be created.
>> One could also manually move replicas as with any other collection,
>> (aside:
>> be sure to refer to a current version of MOVEREPLICA docs, prior to

Re: OutofMemory Error in solr 6.5

[Satya Nand]

Hi Dominique,

Thanks, But I still have one confusion, Please help me with it.

Pretty sure the issue is caused by caches size at new searcher warmup time.


We use leader-follower architecture with a replication interval of 3 hours.
This means every 3 hours we get a commit and the searcher warms up. Right?
We have frequent indexing, It isn't possible that we don't get any document
indexed in 3 hours period.

So the new searcher warms up 7-8 times a day. If the issue was due to a new
searcher warm-up, We would have got the issue many times a day. But we get
this OutOfMemory Issue once or twice a month or sometimes once in two
months, so some combination of events must be triggering it.


Further, In solrConfig we have
2

does this mean that at some time up to 3 searchers(1 active and 2 still
warming up their caches) might be holding to their caches and this
resulting in the issue? Sould we reduce this to 1 ? what impact would it
have if we reduced it ?

On Wed, Aug 11, 2021 at 2:15 AM Dominique Bejean 
wrote:

> Pretty sure the issue is caused by caches size at new searcher warmup time.
>
> Dominique
>
>
> Le mar. 10 août 2021 à 09:07, Satya Nand  a
> écrit :
>
>> Hi Dominique,
>>
>> You don't provide information about the number of documents. Anyway, all
>>> your cache size and mostly initial size are big. Cache are stored in JVM
>>> heap.
>>
>> Document count is 101893353.
>>
>> About cache size, most is not always better. Did you make some performance
>>> benchmarks in order to set these values ?
>>
>> We increased cache size in the hope to reduce some response time, We
>> heavily use group queries with 7-8 boost factors. The average response time
>> on this document set is 136 ms. We receive approx 65 requests/second in
>> peak hours. The replication interval is 3 hours.
>>
>> The most strange thing about is that system keeps running for days
>> without any issue, So I believe cache size should not be an issue. If the
>> cache size had been the culprit, the issue would have been frequent.  isn't
>> it?
>>
>>
>>
>> On Mon, Aug 9, 2021 at 6:44 PM Dominique Bejean <
>> dominique.bej...@eolya.fr> wrote:
>>
>>> Hi,
>>>
>>> You don't provide information about the number of documents. Anyway, all
>>> your cache size and mostly initial size are big. Cache are stored in JVM
>>> heap.
>>>
>>> About cache size, most is not always better. Did you make some
>>> performance
>>> benchmarks in order to set these values ?
>>>
>>> Try with the default values, after a few hours check cumulative caches
>>> statistics in order to decide if you need to increase their sizes. The
>>> objective is not to have cumulative_hitratio to 100%. There isn't ideal
>>> value as it is really related to your datas, to the user's queries, to
>>> how
>>> you build your queries ... but 70% is a good value. At some point
>>> increasing the size again and again won't increase cumulative_hitratio a
>>> lot as it is a logarithmic curve.
>>>
>>> Check also the heap usage also with your JVM GC logs and a tool like
>>> gceasy.io
>>>
>>> Regards
>>>
>>> Dominique
>>>
>>>
>>>
>>>
>>> Le lun. 9 août 2021 à 07:44, Satya Nand >> .invalid>
>>> a écrit :
>>>
>>> > Hi,
>>> > We are facing a strange issue in our solr system. Most of the days it
>>> keeps
>>> > running fine but once or twice in a month, we face OutofMemory on solr
>>> > servers.
>>> >
>>> > We are using Leader-Follower architecture, one Leader and 4 followers.
>>> > Strangely we get OutofMemory error on all follower servers.
>>> > Before the OutOfMemory this exception is found on all servers.
>>> >
>>> > Aug, 04 2021 15:26:11 org.apache.solr.servlet.HttpSolrCall
>>> > search-central-prd-solr-temp1
>>> > ERROR: null:java.lang.NullPointerException
>>> search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:617)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:240)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:2027)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1844)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:609)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:547)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
>>> > search-central-prd-solr-temp1
>>> > at
>>> >
>>> >
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
>>> > search-central-prd-s

Re: OutofMemory Error in solr 6.5

[Satya Nand]

Hi Shawn,

Thanks for explaining it so well. We will work on reducing the filter cache
size and auto warm count.

Though I have one question.

If your configured 4000 entry filterCache were to actually fill up, it
> would require nearly 51 billion bytes, and that's just for the one core
> with 101 million documents.  This is much larger than the 30GB heap you
> have specified ... I am betting that the filterCache is the reason
> you're hitting OOME.


As you can see from the below screenshots the filter cache is almost full
and the heap is approx 18-20 GB. I think this means heap is not actually
taking 51 GB of space. Otherwise, the issue would have been very frequent
if the full cache had been taking ~50 GB of space. I also believed the solr
uses some compressed data structures to accumulate its cache, That' how it
is able to store the cache in less memory. Isn't it?

Also, the issue is not very frequent. It comes once or twice a month, Where
all follower servers stop working at the same time due to OutOfMemory error.

*Filter Cache statics as of 10:08 IST*
[image: image.png]

*Heap Usages*
[image: image.png]

   -







On Wed, Aug 11, 2021 at 4:12 AM Shawn Heisey  wrote:

> On 8/10/2021 1:06 AM, Satya Nand wrote:
> > Document count is 101893353.
>
> The OOME exception confirms that we are dealing with heap memory.  That
> means we won't have to look into the other resource types that can cause
> OOME.
>
> With that document count, each filterCache entry is 12736670 bytes, plus
> some small number of bytes for java object overhead.  That's 12.7
> million bytes.
>
> If your configured 4000 entry filterCache were to actually fill up, it
> would require nearly 51 billion bytes, and that's just for the one core
> with 101 million documents.  This is much larger than the 30GB heap you
> have specified ... I am betting that the filterCache is the reason
> you're hitting OOME.
>
> You need to dramatically reduce the size of your filterCache.  Start
> with 256 and see what that gets you.  Solr ships with a size of 512.
> Also, see what you can do about making it so that there is a lot of
> re-use possible with queries that you put in the fq parameter.  It's
> better to have several fq parameters rather than one parameter with a
> lot of AND clauses -- much more chance of filter re-use.
>
> I notice that you have autowarmCount set to 100 on two caches.  (The
> autowarmCount on the documentCache, which you have set to 512, won't be
> used -- that cache cannot be warmed directly.  It is indirectly warmed
> when the other caches are warmed.)  This means that every time you issue
> a commit that opens a new searcher, Solr will execute up to 200 queries
> as part of the cache warming.  This can make the warming take a VERY
> long time.  Consider reducing autowarmCount.  It's not causing your OOME
> problems, but it might be making commits take a very long time.
>
> Thanks,
> Shawn
>

-- 

Re: Big difference in response time on solr 8.7; Optimized vs un Optimized core

[Deepak Goel]

If I were you, then I would stick to the 128 GB machine. And then look at
other parameters to tune...


Deepak
"The greatness of a nation can be judged by the way its animals are treated
- Mahatma Gandhi"

+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

Make In India : http://www.makeinindia.com/home


On Tue, Aug 3, 2021 at 3:25 PM Satya Nand 
wrote:

> Hi Deepak,
>
> We actually tried with 128 GB machine, Didn't help in response time. So we
> moved back to 96GB.
>
> On Tue, Aug 3, 2021 at 2:11 PM Deepak Goel  wrote:
>
> > I am confused a bit about the maths:
> >
> > Heap-30 GB & Index Size-95 GB is equal to 125GB. And the RAM is 96GB.
> >
> >
> >
> >
> > Deepak
> > "The greatness of a nation can be judged by the way its animals are
> treated
> > - Mahatma Gandhi"
> >
> > +91 73500 12833
> > deic...@gmail.com
> >
> > Facebook: https://www.facebook.com/deicool
> > LinkedIn: www.linkedin.com/in/deicool
> >
> > "Plant a Tree, Go Green"
> >
> > Make In India : http://www.makeinindia.com/home
> >
> >
> > On Tue, Aug 3, 2021 at 12:10 PM Satya Nand  > .invalid>
> > wrote:
> >
> > > Hi,
> > >
> > > We have recently upgraded solr from version 6.5 to version 8.7.  But
> > > opposite to our expectations, the response time increased by 40 %.
> > >
> > > On solr 8.7 the difference between optimized and unoptimized index is
> > also
> > > very huge. 350 ms on optimized and 650 ms on unoptimized. The
> difference
> > is
> > > only 5 GB in size in cores of optimized and unoptimized. The segment
> > count
> > > in the optimized index is 1 and 20 in the unoptimized index.
> > >
> > > I wanted to ask, Is this normal behavior on solr 8.7, or was there some
> > > setting that we forgot to add? Pleas also tell us how can we reduce the
> > > response time in unoptimzed core.
> > >
> > > *Specifications*
> > > We are using master slave architecture, Polling interval is 3 hours
> > > RAM- 96 GB
> > > CPU-14
> > > Heap-30 GB
> > > Index Size-95 GB
> > > Segments size-20
> > > Merge Policy :
> > >
> > >  > class="org.apache.solr.index.TieredMergePolicyFactory">
> > > 5 3
>  > > mergePolicyFactory>
> > >
> > > --
> > >
> > >
> >
>
> --
>
>