Is it possible that there are too frequent commits? I mean if each commit
usually invalidates the cache, even the a stupid *:* rows=1 can be affected.
How can I see how frequent commits are? Or when the latest commit has been
done?
On Fri, Mar 18, 2022 at 8:36 PM Vincenzo D'Amore <v.dam...@gmail.com> wrote:
> Ok, everything you said is right, but nevertheless even right now a stupid
> *:* rows=1 runs in almost 2 seconds.
> The average document size is pretty small, less than roughly 100/200
> bytes.
> Does someone know if the average doc size is available in the metrics?
>
> {
> "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":2033,
> "params":{
> "q":"*:*",
> "rows":"1"}},
>
> On Fri, Mar 18, 2022 at 7:50 PM matthew sporleder <msporle...@gmail.com>
> wrote:
>
>> You are getting this general advice but, sadly, it depends on your doc
>> sizes, query complexity, write frequency, and a bunch of other stuff I
>> don't know about.
>>
>> I prefer to run with the *minimum* JVM memory to handle throughput
>> (without
>> OOM) and let the OS do caching because I update/write to the index every
>> few minutes making my *solr* caching pretty worthless.
>>
>> tuning solr also includes tuning queries. Start with timing id:123 type
>> K:V lookups and work your complexity up from there. Use debug=true and
>> attempt to read it.
>>
>> There are many many knobs. You need to set a baseline, then a target,
>> then
>> a thesis on how to get there.
>>
>>
>>
>> On Fri, Mar 18, 2022 at 2:36 PM Vincenzo D'Amore <v.dam...@gmail.com>
>> wrote:
>>
>> > We have modified the kubernetes configuration and restarted SolrCloud
>> > cluster, now we have 16 cores per Solr instance.
>> > The performance does not seem to be improved though.
>> > The load average is 0.43 0.83 1.00, to me it seems an IO bound problem.
>> > Looking at the index I see 162M documents, 234M maxDocs, 71M deleted...
>> > maybe this core needs to be optimized.
>> > The INDEX.size is 70GB, what do you think if I raise the size allocated
>> > from the JVM to 64GB in order to have the index in memory?
>> > At last, I'm looking at Solr metric but really not sure how to
>> understand
>> > if it is CPU bound or IO bound.
>> >
>> > On Fri, Mar 18, 2022 at 6:34 PM Walter Underwood <wun...@wunderwood.org
>> >
>> > wrote:
>> >
>> > > First look at the system metrics. Is it CPU bound or IO bound? Each
>> > > request is single threaded, so a CPU bound system will have one core
>> used
>> > > at roughly 100% for that time. An IO bound system will not be using
>> much
>> > > CPU but will have threads in iowait and lots of disk reads.
>> > >
>> > > After you know that, then you know what to work on. If it is IO bound,
>> > get
>> > > enough RAM for the OS, JVM, and index files to all be in memory. If
>> it is
>> > > CPU bound, get a faster processor and work on the config to have the
>> > > request do less work. Sharding can also help.
>> > >
>> > > I’m not a fan of always choosing 31 GB for the JVM. Allocate only as
>> much
>> > > as is needed. Java will use the whole heap whether it is needed or
>> not.
>> > You
>> > > might only need 8 GB. All of our clusters run with 16 GB. That
>> includes
>> > > some machines with 36 cores.
>> > >
>> > >
>> > --
>> > Vincenzo D'Amore
>> >
>>
>
>
> --
> Vincenzo D'Amore
>
>
--
Vincenzo D'Amore
