Walter I agree, but with large indexes (850+Gb before merge) I just found 31 to be my happy spot. As well as set xms and xmx to the same value, I have no proof but it seems to take less processing to keep them the same than to keep allocating different memory footprints
> On Mar 18, 2022, at 1:38 PM, Vincenzo D'Amore <v.dam...@gmail.com> wrote: > > I did it right now in prod environment: > { > "responseHeader":{ > "zkConnected":true, > "status":0, > "QTime":1943, > "params":{ > "q":"*:*", > "rows":"1"}}, > > then for a while, the QTime is 0. I assume (obviously) that it is cached, > but after a while the cache expires.... > >> On Fri, Mar 18, 2022 at 6:22 PM Dave <hastings.recurs...@gmail.com> wrote: >> >> I’ve found that each solr instance will take as many cores as it needs per >> request. Your 2 sec response sounds like you just started the server and >> then did that search. I never trust the first search as nothing has been >> put into memory yet. I like to give my jvms 31 gb each and let Linux cache >> the rest of the files as it sees fit, with swap turned completely off. Also >> *:* can be heavier than you think if you have every field indexed since >> it’s like a punch card like system where all the fields have to match. >> >>> On Mar 18, 2022, at 12:45 PM, Vincenzo D'Amore <v.dam...@gmail.com> >> wrote: >>> >>> Thanks for your support, just sharing what I found until now. >>> >>> I'm working with SolrCloud with a 2 node deployment. This deployment has >>> many indexes but a main one 160GB index that has become very slow. >>> Select *:* rows=1 take 2 seconds. >>> SolrCloud instances are running in kubernetes and are deployed in a pod >>> with 128GB RAM but only 16GB to JVM. >>> Looking at Solr Documentation I've found nothing specific about what >>> happens to the performance if the number of CPUs is not correctly >> detected. >>> The only interesting page is the following and it seems to match with >> your >>> suggestion. >>> At the end of paragraph there is a not very clear reference about how the >>> Concurrent Merge Scheduler behavior can be impacted by the number of >>> detected CPUs. >>> >>>> Similarly, the system property lucene.cms.override_core_count can be set >>> to the number of CPU cores to override the auto-detected processor count. >>> >>>> Talking Solr to Production > Dynamic Defaults for >> ConcurrentMergeScheduler >>>> >>> >> https://solr.apache.org/guide/8_3/taking-solr-to-production.html#dynamic-defaults-for-concurrentmergescheduler >>> >>> >>> >>>> On Thu, Mar 17, 2022 at 1:22 PM Thomas Matthijs <li...@selckin.be> >> wrote: >>>> >>>> I don't know how it affects solr, but if you're interested in java's >>>> support to detect cgroup/container limits on cpu/memory etc, you can use >>>> these links as starting points to investigate. >>>> It affect some jvm configuration, like initial GC selection & settings >>>> that can affect performance. >>>> It was only backported to java 8 quite recently, so if you're still on >>>> that might want to check if you're on the latest version. >>>> >>>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8146115 >>>> https://bugs.openjdk.java.net/browse/JDK-8264136 >>>> >>>> >>>>> On Thu, Mar 17, 2022, at 01:11, Vincenzo D'Amore wrote: >>>>> Hi Shawn, thanks for your help. >>>>> >>>>> Given that I’ll put the question in another way. >>>>> If Java don’t correctly detect the number of CPU how the overall >>>>> performance can be affected by this? >>>>> >>>>> Ciao, >>>>> Vincenzo >>>>> >>>>> -- >>>>> mobile: 3498513251 >>>>> skype: free.dev >>>>> >>>>>> On 16 Mar 2022, at 18:56, Shawn Heisey <elyog...@elyograg.org> wrote: >>>>>> >>>>>> On 3/16/22 03:56, Vincenzo D'Amore wrote: >>>>>>> just asking how can I rely on the number of processors the solr >>>> dashboard >>>>>>> shows. >>>>>>> >>>>>>> Just to give you a context, I have a 2 nodes solrcloud instance >>>> running in >>>>>>> kubernetes. >>>>>>> Looking at solr dashboard (8.3.0) I see there is only 1 cpu available >>>> per >>>>>>> solr instance. >>>>>>> but the Solr pods are deployed in two different kube nodes, and >>>> entering >>>>>>> the pod with the >>>>>>> kubectl exec -ti solr-0 -- /bin/bash >>>>>>> and running top I see there are 16 cores available for each solr >>>> instance. >>>>>> >>>>>> The dashboard info comes from Java, and Java gets it from the OS. How >>>> that works with containers is something I don't know much about. Here's >>>> what Linux says about a server I have which has two six-core Intel CPUs >>>> with hyperthreading. This is bare metal, not a VM or container: >>>>>> >>>>>> elyograg@smeagol:~$ grep processor /proc/cpuinfo >>>>>> processor : 0 >>>>>> processor : 1 >>>>>> processor : 2 >>>>>> processor : 3 >>>>>> processor : 4 >>>>>> processor : 5 >>>>>> processor : 6 >>>>>> processor : 7 >>>>>> processor : 8 >>>>>> processor : 9 >>>>>> processor : 10 >>>>>> processor : 11 >>>>>> processor : 12 >>>>>> processor : 13 >>>>>> processor : 14 >>>>>> processor : 15 >>>>>> processor : 16 >>>>>> processor : 17 >>>>>> processor : 18 >>>>>> processor : 19 >>>>>> processor : 20 >>>>>> processor : 21 >>>>>> processor : 22 >>>>>> processor : 23 >>>>>> >>>>>> If I start Solr on that server, the dashboard reports 24 processors. >>>>>> >>>>>> Thanks, >>>>>> Shawn >>>>>> >>>> >>> >>> >>> -- >>> Vincenzo D'Amore >> > > > -- > Vincenzo D'Amore