Tomas,
I just set the max warming searchers to 4, and I still see 0ms on warming time.
Not sure what else to check.
Thanks in advance!
________________________________
From: Matias Laino <matias.la...@passare.com.INVALID>
Sent: Friday, December 9, 2022 12:10 PM
To: Tomás Fernández Löbbe <tomasflo...@gmail.com>; users@solr.apache.org
<users@solr.apache.org>
Subject: Re: Near Real Time not working as expected
Thank you Tomas! This was really useful info, I checked some of my logs for
today... but it say "Registered new searcher autowarm time: 0 ms"
I'm very confused right now lol, it sounds odd to have 0ms to me.
This is my current Cache connfiguration (I removed the maxWarmingSearchers
option as a test):
<filterCache class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<queryResultCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<documentCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
<cache name="perSegFilter"
class="solr.search.LRUCache"
size="10"
initialSize="0"
autowarmCount="10"
regenerator="solr.NoOpRegenerator" />
________________________________
From: Tomás Fernández Löbbe <tomasflo...@gmail.com>
Sent: Thursday, December 8, 2022 8:22 PM
To: users@solr.apache.org <users@solr.apache.org>
Subject: Re: Near Real Time not working as expected
If you see this warning, then the issue is that your warming is taking too
long. Consider:
Reducing/Removing auto-warm[1]. You may also have static warming with query
listeners[2]? If you have INFO logging enabled in SolrCore it should be
printing something like:
"Registered new searcher autowarm time: X ms"
Check those values, with 1s autoSoftCommit you probably want to have
autowarm time to be as close as 0 as possible.
[1]
https://solr.apache.org/guide/solr/latest/configuration-guide/caches-warming.html
[2]
https://solr.apache.org/guide/solr/latest/configuration-guide/caches-warming.html#query-related-listeners
On Thu, Dec 8, 2022 at 6:51 AM Matias Laino
<matias.la...@passare.com.invalid> wrote:
> Hi Tomas!
> Yes! I saw that message, my original setting for auto warm searchers was
> 2, I increased it to 6 and I was still seeing the message, now it's at to
> 16 and I don't see that ( I went from 6 to 16), but the issue still
> persists.
>
> I havent seen post commit events, where can I look for that ? Sorry, I'm
> relatively novice on configuring solr from scratch.
>
> Thanks in advance!
>
> ________________________________
> From: Tomás Fernández Löbbe <tomasflo...@gmail.com>
> Sent: Wednesday, December 7, 2022 6:56 PM
> To: users@solr.apache.org <users@solr.apache.org>
> Subject: Re: Near Real Time not working as expected
>
> Are you seeing any messages in the logs with "PERFORMANCE WARNING:
> Overlapping onDeckSearchers"? Can you elaborate on the autowarm
> configuration that you have? any "postCommit" events?
>
> If you set the logger of "org.apache.solr.search.SolrIndexSearcher" to
> DEBUG level you should see when the searcher is open and how long it takes
> to warmup.
>
>
> On Wed, Dec 7, 2022 at 9:58 AM Matias Laino
> <matias.la...@passare.com.invalid> wrote:
>
> > I'm sorry but I'm not sure what you mean with metal, our servers are EC2
> > instances if that helps in any way.
> >
> > MATIAS LAINO | DIRECTOR OF PASSARE REMOTE DEVELOPMENT
> > matias.la...@passare.com | +54 11-6357-2143
> >
> >
> > -----Original Message-----
> > From: Dave <hastings.recurs...@gmail.com>
> > Sent: Wednesday, December 7, 2022 2:40 PM
> > To: users@solr.apache.org
> > Subject: Re: Near Real Time not working as expected
> >
> > Just out of curiosity are you using metal? And if so ran any disk io
> tests
> > to see if you may have a hardware problem on any of the nodes? A
> document
> > won’t be available until all the nodes have it so it just takes one to
> get
> > slow to slow you down
> >
> > > On Dec 7, 2022, at 9:45 AM, Matias Laino <matias.la...@passare.com
> .invalid>
> > wrote:
> > >
> > >
> > > Hi all,
> > >
> > > I recently had an issue with very high cpu usage on our Testing
> > SolrCloud cluster when sending data to Solr, I’ve tried several which
> > reduced the usage of CPU, now our testing SolrCloud is under an 8 core
> > machine with 32 gb de RAM (recently changed the heap to 21g as a test).
> > > When we push data to solr, it takes a couple of minutes for that
> > document to be available on search results, I’ve tried everything and
> > cannot find out what is going on, it was working perfectly fine until
> last
> > week when it suddenly started having this delay.
> > >
> > > Our configuration for NRT is very aggressive, 60s of auto commit with
> > open searcher false and 1s for auto soft commit, but it doesn’t matter
> what
> > configuration I try, it will always take a couple of minutes to have the
> > new document available on search results.
> > >
> > > I’ve tried modifying the cache configuration to use Caffeine, tried
> > removing max warming searchers values, tried modifying autoWarmCount to
> > different values and even tried, and still the same issue, it’s almost
> like
> > my configuration doesn’t matter.
> > >
> > > We are using a Solr 8.11 install in SolrCloud mode, 2 nodes, 1
> Zookeeper
> > node. On each node we have 6 collections of around 10-11M records each
> > (numbers didn’t change much before and after this issue started). The
> total
> > amount of disk spaced used is 20.4gb, our heap is now 21gb.
> > >
> > > I’m kind of desperate since I’ll be on vacation starting the end of
> next
> > week and I haven’t been able to find out what is wrong with this, my fear
> > is if this happens to our production server, we won’t be able to know how
> > to fix it other than reinstalling Solr from scratch.
> > >
> > > Our prod server only has 1 collection of 11gb and is under a 4 core
> > servers with 16 gb of ram (8gb heap setup).
> > >
> > > Any help or pointer will be highly appreciated as I’m desperate.
> > >
> > > Thanks in advance!
> > >
> > > MATIAS LAINO | DIRECTOR OF PASSARE REMOTE DEVELOPMENT
> > > matias.la...@passare.com | +54 11-6357-2143
> > >
> > >
> >
>
