It may sound counter intuitive, but allocate as little Java heap as possible to Solr, without causing OOM. Read up on the reference guide links provided as well as excellent advice on profiling your heap usage.
Jan Høydahl > 14. mar. 2023 kl. 15:03 skrev HariBabu kuruva <hari2708.kur...@gmail.com>: > > Hi , > > Till now it was running with 45GB heap memory. I am trying to tune the > performance of solr by adjusting heap memory. > > So, I am looking for your inputs. > >> On Tue, Mar 14, 2023 at 3:23 PM Jan Høydahl <jan....@cominvent.com> wrote: >> >> Why do you believe you need such a huge heap as 31g? Can you support such >> a choice by some observations or measurements? >> >> Jan >> >>> 14. mar. 2023 kl. 06:39 skrev HariBabu kuruva <hari2708.kur...@gmail.com >>> : >>> >>> Thank you all for your responses. >>> >>> There are no spaces between Xms and the values. >>> >>> I have updated similar arguments(-Xms30720m -Xmx30720m) in one of the >>> non-prod environments(in mbs instead of gb). It correctly shows the max >>> heap as 30GB in the Solr UI. >>> >>> So, I would like to update 31.5 GB similarly in PROD (-Xms32256m >>> -Xmx32256m). Before applying that should i analyze heap memory stats >> using >>> Heap analyzer tool or i can go ahead and apply. I have a total server >>> memory of 62GB and only solr is running in it. Please suggest. >>> >>> >>> On Thu, Mar 9, 2023 at 8:00 PM Walter Underwood <wun...@wunderwood.org> >>> wrote: >>> >>>> Use a heap analysis tool. You’ll see a sawtooth pattern in the heap >> size. >>>> The bottom of that sawtooth is the actual amount of memory that Solr is >>>> using. Pick the highest point of the bottom of the sawtooth, then add >> some >>>> headroom, maybe a gigabyte. Test with that value. >>>> >>>> wunder >>>> Walter Underwood >>>> wun...@wunderwood.org >>>> https://observer.wunderwood.org/ (my blog) >>>> >>>>> On Mar 9, 2023, at 5:23 AM, Dave <hastings.recurs...@gmail.com> wrote: >>>>> >>>>> Agreed, but often times as a developer you are subject to the requests >>>> of those higher up and you end up with 30 facets of strings that are the >>>> length of names. But yes, test as low as you can and try to keep the >>>> qtimes low and just keep adjusting until you are happy with whatever >> time >>>> works for you, I preferred sub half second but the index was 400+ gb so >> I >>>> used a large heap >>>>> >>>>>> On Mar 9, 2023, at 7:56 AM, Jan Høydahl <jan....@cominvent.com> >> wrote: >>>>>> >>>>>> It's a waste to set heap to 30g if your use of Solr only requires 6g >>>> to function. That is 24G memory not being used for index caching, and it >>>> will may, depending on chose GC, cause bigger/longer GC events as more >>>> garbage piles up before collection. >>>>>> >>>>>> You have to measure and experiment to find your heap sweetspot. >>>>>> It is a common misconception that giving more heap to Solr improves >>>> performance of something, it's rather the contrary. Only rarely do you >> need >>>> huge internal solr caches, and if you find yourself needing that kind of >>>> memory due to requesting huge number of docs or faceting on high >>>> cardinality long string fields or whatever, then it is better to >> optimize >>>> your use of Solr to lower the footprint than to add >30g. >>>>>> >>>>>> Jan >>>>>> >>>>>>> 9. mar. 2023 kl. 12:52 skrev Dave <hastings.recurs...@gmail.com>: >>>>>>> >>>>>>> Again, set to less than 32, I liked 30 >>>>>>> >>>>>>>>> On Mar 9, 2023, at 1:04 AM, Deepak Goel <deic...@gmail.com> wrote: >>>>>>>> >>>>>>>> The max heap could be the max heap used by the process uptill now. >>>> And not >>>>>>>> the max value you have set. I would suggest you increase the load by >>>> at >>>>>>>> least 20 times to see the max heap to go to 32 gb. >>>>>>>> >>>>>>>> >>>>>>>> Deepak >>>>>>>> "The greatness of a nation can be judged by the way its animals are >>>> treated >>>>>>>> - Mahatma Gandhi" >>>>>>>> >>>>>>>> +91 73500 12833 >>>>>>>> deic...@gmail.com >>>>>>>> >>>>>>>> Facebook: https://www.facebook.com/deicool >>>>>>>> LinkedIn: www.linkedin.com/in/deicool >>>>>>>> >>>>>>>> "Plant a Tree, Go Green" >>>>>>>> >>>>>>>> Make In India : http://www.makeinindia.com/home >>>>>>>> >>>>>>>> >>>>>>>>> On Thu, Mar 9, 2023 at 8:21 AM Shawn Heisey <apa...@elyograg.org> >>>> wrote: >>>>>>>>> >>>>>>>>>> On 3/8/2023 9:24 AM, HariBabu kuruva wrote: >>>>>>>>>> I have set the Heap memory as -Xms 1g -Xmx 40g in the Production >>>>>>>>>> environment. >>>>>>>>>> >>>>>>>>>> But when i see the Heap memory in the Solr UI. I can see the Max >>>> Heap >>>>>>>>> below. >>>>>>>>>> >>>>>>>>>> Max: 3.8Gb >>>>>>>>>> Used: 2.2Gb >>>>>>>>> >>>>>>>>> The other answers you've gotten are good. This is mostly just a >>>> little >>>>>>>>> more detail. >>>>>>>>> >>>>>>>>> If you really do have the spaces before the 1g and 40g, then those >>>>>>>>> values are probably not being honored as Jan said. >>>>>>>>> >>>>>>>>> If you don't have the spaces, then Java will start with the heap >>>> size at >>>>>>>>> 1GB, and only increase it if there is enough memory pressure. One >>>> thing >>>>>>>>> that I don't know is whether Java will use the 32 bit pointers with >>>> the >>>>>>>>> Xmx at 40g. It probably won't, so I expect that memory usage would >>>> be >>>>>>>>> more efficient if you set the max heap to 31g. >>>>>>>>> >>>>>>>>> With a heap size at 32GB or larger, Java has to use 64 bit >> pointers, >>>> and >>>>>>>>> that will make it use quite a bit more memory because Solr creates >> a >>>> LOT >>>>>>>>> of objects. >>>>>>>>> >>>>>>>>> As Dave said, if you use Xms and Xmx, then they should be set to >> the >>>>>>>>> same value. And the value should be less than 32GB for efficiency. >>>>>>>>> >>>>>>>>> If you use the SOLR_HEAP environment variable in the include >> script, >>>>>>>>> then both Xms and Xmx will be set to the provided value. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Shawn >>>>>>>>> >>>>>> >>>> >>>> >>> >>> -- >>> >>> Thanks and Regards, >>> Hari >>> Mobile:9790756568 >> >> > > -- > > Thanks and Regards, > Hari > Mobile:9790756568
