Walter I agree, but with large indexes (850+Gb before merge) I just found 31 to
be my happy spot. As well as set xms and xmx to the same value, I have no
proof but it seems to take less processing to keep them the same than to keep
allocating different memory footprints
> On Mar 18, 2022, at 1:38 PM, Vincenzo D'Amore <v.dam...@gmail.com> wrote:
>
> I did it right now in prod environment:
> {
> "responseHeader":{
> "zkConnected":true,
> "status":0,
> "QTime":1943,
> "params":{
> "q":"*:*",
> "rows":"1"}},
>
> then for a while, the QTime is 0. I assume (obviously) that it is cached,
> but after a while the cache expires....
>
>> On Fri, Mar 18, 2022 at 6:22 PM Dave <hastings.recurs...@gmail.com> wrote:
>>
>> I’ve found that each solr instance will take as many cores as it needs per
>> request. Your 2 sec response sounds like you just started the server and
>> then did that search. I never trust the first search as nothing has been
>> put into memory yet. I like to give my jvms 31 gb each and let Linux cache
>> the rest of the files as it sees fit, with swap turned completely off. Also
>> *:* can be heavier than you think if you have every field indexed since
>> it’s like a punch card like system where all the fields have to match.
>>
>>> On Mar 18, 2022, at 12:45 PM, Vincenzo D'Amore <v.dam...@gmail.com>
>> wrote:
>>>
>>> Thanks for your support, just sharing what I found until now.
>>>
>>> I'm working with SolrCloud with a 2 node deployment. This deployment has
>>> many indexes but a main one 160GB index that has become very slow.
>>> Select *:* rows=1 take 2 seconds.
>>> SolrCloud instances are running in kubernetes and are deployed in a pod
>>> with 128GB RAM but only 16GB to JVM.
>>> Looking at Solr Documentation I've found nothing specific about what
>>> happens to the performance if the number of CPUs is not correctly
>> detected.
>>> The only interesting page is the following and it seems to match with
>> your
>>> suggestion.
>>> At the end of paragraph there is a not very clear reference about how the
>>> Concurrent Merge Scheduler behavior can be impacted by the number of
>>> detected CPUs.
>>>
>>>> Similarly, the system property lucene.cms.override_core_count can be set
>>> to the number of CPU cores to override the auto-detected processor count.
>>>
>>>> Talking Solr to Production > Dynamic Defaults for
>> ConcurrentMergeScheduler
>>>>
>>>
>> https://solr.apache.org/guide/8_3/taking-solr-to-production.html#dynamic-defaults-for-concurrentmergescheduler
>>>
>>>
>>>
>>>> On Thu, Mar 17, 2022 at 1:22 PM Thomas Matthijs <li...@selckin.be>
>> wrote:
>>>>
>>>> I don't know how it affects solr, but if you're interested in java's
>>>> support to detect cgroup/container limits on cpu/memory etc, you can use
>>>> these links as starting points to investigate.
>>>> It affect some jvm configuration, like initial GC selection & settings
>>>> that can affect performance.
>>>> It was only backported to java 8 quite recently, so if you're still on
>>>> that might want to check if you're on the latest version.
>>>>
>>>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8146115
>>>> https://bugs.openjdk.java.net/browse/JDK-8264136
>>>>
>>>>
>>>>> On Thu, Mar 17, 2022, at 01:11, Vincenzo D'Amore wrote:
>>>>> Hi Shawn, thanks for your help.
>>>>>
>>>>> Given that I’ll put the question in another way.
>>>>> If Java don’t correctly detect the number of CPU how the overall
>>>>> performance can be affected by this?
>>>>>
>>>>> Ciao,
>>>>> Vincenzo
>>>>>
>>>>> --
>>>>> mobile: 3498513251
>>>>> skype: free.dev
>>>>>
>>>>>> On 16 Mar 2022, at 18:56, Shawn Heisey <elyog...@elyograg.org> wrote:
>>>>>>
>>>>>> On 3/16/22 03:56, Vincenzo D'Amore wrote:
>>>>>>> just asking how can I rely on the number of processors the solr
>>>> dashboard
>>>>>>> shows.
>>>>>>>
>>>>>>> Just to give you a context, I have a 2 nodes solrcloud instance
>>>> running in
>>>>>>> kubernetes.
>>>>>>> Looking at solr dashboard (8.3.0) I see there is only 1 cpu available
>>>> per
>>>>>>> solr instance.
>>>>>>> but the Solr pods are deployed in two different kube nodes, and
>>>> entering
>>>>>>> the pod with the
>>>>>>> kubectl exec -ti solr-0 -- /bin/bash
>>>>>>> and running top I see there are 16 cores available for each solr
>>>> instance.
>>>>>>
>>>>>> The dashboard info comes from Java, and Java gets it from the OS. How
>>>> that works with containers is something I don't know much about. Here's
>>>> what Linux says about a server I have which has two six-core Intel CPUs
>>>> with hyperthreading. This is bare metal, not a VM or container:
>>>>>>
>>>>>> elyograg@smeagol:~$ grep processor /proc/cpuinfo
>>>>>> processor : 0
>>>>>> processor : 1
>>>>>> processor : 2
>>>>>> processor : 3
>>>>>> processor : 4
>>>>>> processor : 5
>>>>>> processor : 6
>>>>>> processor : 7
>>>>>> processor : 8
>>>>>> processor : 9
>>>>>> processor : 10
>>>>>> processor : 11
>>>>>> processor : 12
>>>>>> processor : 13
>>>>>> processor : 14
>>>>>> processor : 15
>>>>>> processor : 16
>>>>>> processor : 17
>>>>>> processor : 18
>>>>>> processor : 19
>>>>>> processor : 20
>>>>>> processor : 21
>>>>>> processor : 22
>>>>>> processor : 23
>>>>>>
>>>>>> If I start Solr on that server, the dashboard reports 24 processors.
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>>>>
>>>>
>>>
>>>
>>> --
>>> Vincenzo D'Amore
>>
>
>
> --
> Vincenzo D'Amore
