Hi all, Its me again. Anyway, I did a little research and we tried different things and well, some questions I want to ask and some things that I found.
Well after monitoring my system with VirtualVM, I found that GC jumping is from 0.5GB to 2.5GB and it has 4GB of memory for now, so it should not be an issue anymore or what? But will observe it a bit as it might rise I guess a bit. Next thing we found or are thinking about is that writing on a disk might be an issue, we turned off the indexing and some other stuff, but I would say, it did not save much still. I also did go through all the schema fields, not that much really. They are all docValues=true. Also I must say they are all automatically generated, so no manual working there except one field, but this also has docValue=true. Just curious, if the field is not a string/text, can it be docValue=false or still better to have true? And as for uninversion, then we are not using much facets nor other specific things in query, just simple queries. Though I must say we are updating documents quite a bunch, but usage of CPU for being so high, not sure about that. Older version seemed not using CPU so much... I am a bit running out of ideas and hoping that this will continue to work, but I dont like the CPU usage even over night, when nobody uses it. We will try to figure out the issue here and I hope I can ask more questions when in doubt or out of ideas. Also I must admit, solr is really new for me personally. Jaan -----Original Message----- From: Walter Underwood <wun...@wunderwood.org> Sent: 27 October 2020 18:44 To: solr-user@lucene.apache.org Subject: Re: SOLR uses too much CPU and GC is also weird on Windows server That first graph shows a JVM that does not have enough heap for the program it is running. Look at the bottom of the dips. That is the amount of memory still in use after a full GC. You want those dips to drop to about half of the available heap, so I’d immediately increase that heap to 4G. That might not be enough, so you’ll need to to watch that graph after the increase. I’ve been using 8G heaps with Solr since version 1.2. We run this config with Java 8 on over 100 machines. We do not do any faceting, which can take more memory. SOLR_HEAP=8g # Use G1 GC -- wunder 2017-01-23 # Settings from https://wiki.apache.org/solr/ShawnHeisey GC_TUNE=" \ -XX:+UseG1GC \ -XX:+ParallelRefProcEnabled \ -XX:G1HeapRegionSize=8m \ -XX:MaxGCPauseMillis=200 \ -XX:+UseLargePages \ -XX:+AggressiveOpts \ " wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Oct 27, 2020, at 12:48 AM, Jaan Arjasepp <jaan.arjas...@coop.ee> wrote: > > Hello, > > We have been using SOLR for quite some time. We used 6.0 and now we did a > little upgrade to our system and servers and we started to use 8.6.1. > We use it on a Windows Server 2019. > Java version is 11 > Basically using it in a default setting, except giving SOLR 2G of heap. It > used 512, but it ran out of memory and stopped responding. Not sure if it was > the issue. When older version, it managed fine with 512MB. > SOLR is not in a cloud mode, but in solo mode as we use it internally and it > does not have too many request nor indexing actually. > Document sizes are not big, I guess. We only use one core. > Document stats are here: > Num Docs: 3627341 > Max Doc: 4981019 > Heap Memory Usage: 434400 > Deleted Docs: 1353678 > Version: 15999036 > Segment Count: 30 > > The size of index is 2.66GB > > While making upgrade we had to modify one field and a bit of code that uses > it. Thats basically it. It works. > If needed more information about background of the system, I am happy to help. > > > But now to the issue I am having. > If SOLR is started, at first 40-60 minutes it works just fine. CPU is not > high, heap usage seem normal. All is good, but then suddenly, the heap usage > goes crazy, going up and down, up and down and CPU rises to 50-60% of the > usage. Also I noticed over the weekend, when there are no writing usage, the > CPU remains low and decent. I can try it this weekend again to see if and how > this works out. > Also it seems to me, that after 4-5 days of working like this, it stops > responding, but needs to be confirmed with more heap also. > > Heap memory usage via JMX and jconsole -> > https://drive.google.com/file/d/1Zo3B_xFsrrt-WRaxW-0A0QMXDNscXYih/view > ?usp=sharing As you can see, it starts of normal, but then goes crazy > and it has been like this over night. > > This is overall monitoring graphs, as you can see CPU is working hard > or hardly working. -> > https://drive.google.com/file/d/1_Gtz-Bi7LUrj8UZvKfmNMr-8gF_lM2Ra/view > ?usp=sharing VM summary can be found here -> > https://drive.google.com/file/d/1FvdCz0N5pFG1fmX_5OQ2855MVkaL048w/view > ?usp=sharing And finally to have better and quick overview of the SOLR > executing parameters that I have -> > https://drive.google.com/file/d/10VCtYDxflJcvb1aOoxt0u3Nb5JzTjrAI/view > ?usp=sharing > > If you can point me what I have to do to make it work, then I appreciate it a > lot. > > Thank you in advance. > > Best regards, > Jaan > >
