Walter I agree, but with large indexes (850+Gb before merge) I just found 31 to 
be my happy spot.  As well as set xms and xmx to the same value, I have no 
proof but it seems to take less processing to keep them the same than to keep 
allocating different memory footprints 

> On Mar 18, 2022, at 1:38 PM, Vincenzo D'Amore <v.dam...@gmail.com> wrote:
> 
> I did it right now in prod environment:
> {
>  "responseHeader":{
>    "zkConnected":true,
>    "status":0,
>    "QTime":1943,
>    "params":{
>      "q":"*:*",
>      "rows":"1"}},
> 
> then for a while, the QTime is 0. I assume (obviously) that it is cached,
> but after a while the cache expires....
> 
>> On Fri, Mar 18, 2022 at 6:22 PM Dave <hastings.recurs...@gmail.com> wrote:
>> 
>> I’ve found that each solr instance will take as many cores as it needs per
>> request. Your 2 sec response sounds like you just started the server and
>> then did that search. I never trust the first search as nothing has been
>> put into memory yet. I like to give my jvms 31 gb each and let Linux cache
>> the rest of the files as it sees fit, with swap turned completely off. Also
>> *:* can be heavier than you think if you have every field indexed since
>> it’s like a punch card like system where all the fields have to match.
>> 
>>> On Mar 18, 2022, at 12:45 PM, Vincenzo D'Amore <v.dam...@gmail.com>
>> wrote:
>>> 
>>> Thanks for your support, just sharing what I found until now.
>>> 
>>> I'm working with SolrCloud with a 2 node deployment. This deployment has
>>> many indexes but a main one 160GB index that has become very slow.
>>> Select *:* rows=1 take 2 seconds.
>>> SolrCloud instances are running in kubernetes and are deployed in a pod
>>> with 128GB RAM but only 16GB to JVM.
>>> Looking at Solr Documentation I've found nothing specific about what
>>> happens to the performance if the number of CPUs is not correctly
>> detected.
>>> The only interesting page is the following and it seems to match with
>> your
>>> suggestion.
>>> At the end of paragraph there is a not very clear reference about how the
>>> Concurrent Merge Scheduler behavior can be impacted by the number of
>>> detected CPUs.
>>> 
>>>> Similarly, the system property lucene.cms.override_core_count can be set
>>> to the number of CPU cores to override the auto-detected processor count.
>>> 
>>>> Talking Solr to Production > Dynamic Defaults for
>> ConcurrentMergeScheduler
>>>> 
>>> 
>> https://solr.apache.org/guide/8_3/taking-solr-to-production.html#dynamic-defaults-for-concurrentmergescheduler
>>> 
>>> 
>>> 
>>>> On Thu, Mar 17, 2022 at 1:22 PM Thomas Matthijs <li...@selckin.be>
>> wrote:
>>>> 
>>>> I don't know how it affects solr, but if you're interested in java's
>>>> support to detect cgroup/container limits on cpu/memory etc, you can use
>>>> these links as starting points to investigate.
>>>> It affect some jvm configuration, like initial GC selection & settings
>>>> that can affect performance.
>>>> It was only backported to java 8 quite recently, so if you're still on
>>>> that might want to check if you're on the latest version.
>>>> 
>>>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8146115
>>>> https://bugs.openjdk.java.net/browse/JDK-8264136
>>>> 
>>>> 
>>>>> On Thu, Mar 17, 2022, at 01:11, Vincenzo D'Amore wrote:
>>>>> Hi Shawn, thanks for your help.
>>>>> 
>>>>> Given that I’ll put the question in another way.
>>>>> If Java don’t correctly detect the number of CPU how the overall
>>>>> performance can be affected by this?
>>>>> 
>>>>> Ciao,
>>>>> Vincenzo
>>>>> 
>>>>> --
>>>>> mobile: 3498513251
>>>>> skype: free.dev
>>>>> 
>>>>>> On 16 Mar 2022, at 18:56, Shawn Heisey <elyog...@elyograg.org> wrote:
>>>>>> 
>>>>>> On 3/16/22 03:56, Vincenzo D'Amore wrote:
>>>>>>> just asking how can I rely on the number of processors the solr
>>>> dashboard
>>>>>>> shows.
>>>>>>> 
>>>>>>> Just to give you a context, I have a 2 nodes solrcloud instance
>>>> running in
>>>>>>> kubernetes.
>>>>>>> Looking at solr dashboard (8.3.0) I see there is only 1 cpu available
>>>> per
>>>>>>> solr instance.
>>>>>>> but the Solr pods are deployed in two different kube nodes, and
>>>> entering
>>>>>>> the pod with the
>>>>>>> kubectl exec -ti solr-0  -- /bin/bash
>>>>>>> and running top I see there are 16 cores available for each solr
>>>> instance.
>>>>>> 
>>>>>> The dashboard info comes from Java, and Java gets it from the OS. How
>>>> that works with containers is something I don't know much about.  Here's
>>>> what Linux says about a server I have which has two six-core Intel CPUs
>>>> with hyperthreading.  This is bare metal, not a VM or container:
>>>>>> 
>>>>>> elyograg@smeagol:~$ grep processor /proc/cpuinfo
>>>>>> processor    : 0
>>>>>> processor    : 1
>>>>>> processor    : 2
>>>>>> processor    : 3
>>>>>> processor    : 4
>>>>>> processor    : 5
>>>>>> processor    : 6
>>>>>> processor    : 7
>>>>>> processor    : 8
>>>>>> processor    : 9
>>>>>> processor    : 10
>>>>>> processor    : 11
>>>>>> processor    : 12
>>>>>> processor    : 13
>>>>>> processor    : 14
>>>>>> processor    : 15
>>>>>> processor    : 16
>>>>>> processor    : 17
>>>>>> processor    : 18
>>>>>> processor    : 19
>>>>>> processor    : 20
>>>>>> processor    : 21
>>>>>> processor    : 22
>>>>>> processor    : 23
>>>>>> 
>>>>>> If I start Solr on that server, the dashboard reports 24 processors.
>>>>>> 
>>>>>> Thanks,
>>>>>> Shawn
>>>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> Vincenzo D'Amore
>> 
> 
> 
> -- 
> Vincenzo D'Amore

Reply via email to