Hi , Till now it was running with 45GB heap memory. I am trying to tune the performance of solr by adjusting heap memory.
So, I am looking for your inputs. On Tue, Mar 14, 2023 at 3:23 PM Jan Høydahl <jan....@cominvent.com> wrote: > Why do you believe you need such a huge heap as 31g? Can you support such > a choice by some observations or measurements? > > Jan > > > 14. mar. 2023 kl. 06:39 skrev HariBabu kuruva <hari2708.kur...@gmail.com > >: > > > > Thank you all for your responses. > > > > There are no spaces between Xms and the values. > > > > I have updated similar arguments(-Xms30720m -Xmx30720m) in one of the > > non-prod environments(in mbs instead of gb). It correctly shows the max > > heap as 30GB in the Solr UI. > > > > So, I would like to update 31.5 GB similarly in PROD (-Xms32256m > > -Xmx32256m). Before applying that should i analyze heap memory stats > using > > Heap analyzer tool or i can go ahead and apply. I have a total server > > memory of 62GB and only solr is running in it. Please suggest. > > > > > > On Thu, Mar 9, 2023 at 8:00 PM Walter Underwood <wun...@wunderwood.org> > > wrote: > > > >> Use a heap analysis tool. You’ll see a sawtooth pattern in the heap > size. > >> The bottom of that sawtooth is the actual amount of memory that Solr is > >> using. Pick the highest point of the bottom of the sawtooth, then add > some > >> headroom, maybe a gigabyte. Test with that value. > >> > >> wunder > >> Walter Underwood > >> wun...@wunderwood.org > >> https://observer.wunderwood.org/ (my blog) > >> > >>> On Mar 9, 2023, at 5:23 AM, Dave <hastings.recurs...@gmail.com> wrote: > >>> > >>> Agreed, but often times as a developer you are subject to the requests > >> of those higher up and you end up with 30 facets of strings that are the > >> length of names. But yes, test as low as you can and try to keep the > >> qtimes low and just keep adjusting until you are happy with whatever > time > >> works for you, I preferred sub half second but the index was 400+ gb so > I > >> used a large heap > >>> > >>>> On Mar 9, 2023, at 7:56 AM, Jan Høydahl <jan....@cominvent.com> > wrote: > >>>> > >>>> It's a waste to set heap to 30g if your use of Solr only requires 6g > >> to function. That is 24G memory not being used for index caching, and it > >> will may, depending on chose GC, cause bigger/longer GC events as more > >> garbage piles up before collection. > >>>> > >>>> You have to measure and experiment to find your heap sweetspot. > >>>> It is a common misconception that giving more heap to Solr improves > >> performance of something, it's rather the contrary. Only rarely do you > need > >> huge internal solr caches, and if you find yourself needing that kind of > >> memory due to requesting huge number of docs or faceting on high > >> cardinality long string fields or whatever, then it is better to > optimize > >> your use of Solr to lower the footprint than to add >30g. > >>>> > >>>> Jan > >>>> > >>>>> 9. mar. 2023 kl. 12:52 skrev Dave <hastings.recurs...@gmail.com>: > >>>>> > >>>>> Again, set to less than 32, I liked 30 > >>>>> > >>>>>>> On Mar 9, 2023, at 1:04 AM, Deepak Goel <deic...@gmail.com> wrote: > >>>>>> > >>>>>> The max heap could be the max heap used by the process uptill now. > >> And not > >>>>>> the max value you have set. I would suggest you increase the load by > >> at > >>>>>> least 20 times to see the max heap to go to 32 gb. > >>>>>> > >>>>>> > >>>>>> Deepak > >>>>>> "The greatness of a nation can be judged by the way its animals are > >> treated > >>>>>> - Mahatma Gandhi" > >>>>>> > >>>>>> +91 73500 12833 > >>>>>> deic...@gmail.com > >>>>>> > >>>>>> Facebook: https://www.facebook.com/deicool > >>>>>> LinkedIn: www.linkedin.com/in/deicool > >>>>>> > >>>>>> "Plant a Tree, Go Green" > >>>>>> > >>>>>> Make In India : http://www.makeinindia.com/home > >>>>>> > >>>>>> > >>>>>>> On Thu, Mar 9, 2023 at 8:21 AM Shawn Heisey <apa...@elyograg.org> > >> wrote: > >>>>>>> > >>>>>>>> On 3/8/2023 9:24 AM, HariBabu kuruva wrote: > >>>>>>>> I have set the Heap memory as -Xms 1g -Xmx 40g in the Production > >>>>>>>> environment. > >>>>>>>> > >>>>>>>> But when i see the Heap memory in the Solr UI. I can see the Max > >> Heap > >>>>>>> below. > >>>>>>>> > >>>>>>>> Max: 3.8Gb > >>>>>>>> Used: 2.2Gb > >>>>>>> > >>>>>>> The other answers you've gotten are good. This is mostly just a > >> little > >>>>>>> more detail. > >>>>>>> > >>>>>>> If you really do have the spaces before the 1g and 40g, then those > >>>>>>> values are probably not being honored as Jan said. > >>>>>>> > >>>>>>> If you don't have the spaces, then Java will start with the heap > >> size at > >>>>>>> 1GB, and only increase it if there is enough memory pressure. One > >> thing > >>>>>>> that I don't know is whether Java will use the 32 bit pointers with > >> the > >>>>>>> Xmx at 40g. It probably won't, so I expect that memory usage would > >> be > >>>>>>> more efficient if you set the max heap to 31g. > >>>>>>> > >>>>>>> With a heap size at 32GB or larger, Java has to use 64 bit > pointers, > >> and > >>>>>>> that will make it use quite a bit more memory because Solr creates > a > >> LOT > >>>>>>> of objects. > >>>>>>> > >>>>>>> As Dave said, if you use Xms and Xmx, then they should be set to > the > >>>>>>> same value. And the value should be less than 32GB for efficiency. > >>>>>>> > >>>>>>> If you use the SOLR_HEAP environment variable in the include > script, > >>>>>>> then both Xms and Xmx will be set to the provided value. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Shawn > >>>>>>> > >>>> > >> > >> > > > > -- > > > > Thanks and Regards, > > Hari > > Mobile:9790756568 > > -- Thanks and Regards, Hari Mobile:9790756568
