Re: Memory tuning advice for small single-node Solr instance

2024-05-26 Thread Shawn Heisey
On 5/25/24 07:09, Imran Chaudhry wrote: Just a follow-up that since limiting my Solr QuerySet rows to 1000 like this: qs.get_results(rows=1000) ...I have not had a single OOM killer issue. I consider this resolved. Reducing rows can reduce the amount of heap memory that Solr needs because So

Re: Memory tuning advice for small single-node Solr instance

2024-05-25 Thread Imran Chaudhry
Just a follow-up that since limiting my Solr QuerySet rows to 1000 like this: qs.get_results(rows=1000) ...I have not had a single OOM killer issue. I consider this resolved. side-note: I tuned the heap and Java mem by observing the Solr dashboard and observing the "JVM-Memory" grows with variou

Re: Memory tuning advice for small single-node Solr instance

2024-05-09 Thread Imran Chaudhry
On Wed, 8 May 2024 at 17:53, Rajani M wrote: > >I think I have limited the documents returned in each query to 10,000 via > the client software. > > Did you mean the solr query rows param is limited to 10,000? Fetching a > large number of records can be inefficient, it should be less than 100, a

Re: Memory tuning advice for small single-node Solr instance

2024-05-08 Thread Robi Petersen
Yes what Ragani said, try pagination of your results! On Wed, May 8, 2024, 9:53 AM Rajani M wrote: > >I think I have limited the documents returned in each query to 10,000 via > the client software. > > Did you mean the solr query rows param is limited to 10,000? Fetching a > large number of re

Re: Memory tuning advice for small single-node Solr instance

2024-05-08 Thread Rajani M
>I think I have limited the documents returned in each query to 10,000 via the client software. Did you mean the solr query rows param is limited to 10,000? Fetching a large number of records can be inefficient, it should be less than 100, a standard page size. Go through this[1] doc. Hope it help

Re: Memory tuning advice for small single-node Solr instance

2024-05-08 Thread Dmitri Maziuk
On 5/8/24 01:25, Imran Chaudhry wrote: Yes. My thought was that tuning the Solr memory related parameters from the default in an intelligent way may help me "get away" with the limited amount of RAM I have. No. Your VM is underprovisioned for what you're running on it. You could dig into cgro

Re: Memory tuning advice for small single-node Solr instance

2024-05-07 Thread Imran Chaudhry
Hello Rajani, On Tue, May 7, 2024, 1:15 AM Rajani M wrote: > What type of queries do you send to Solr, what is the QTime returned per > query on an average, and how many queries per second/minute? They are 99% read queries with faceting. I am not sure of queries/sec. I don't have hard stats.

Re: Memory tuning advice for small single-node Solr instance

2024-05-07 Thread Imran Chaudhry
Hello Dmitri, On Tue, May 7, 2024, 3:16 PM Dmitri Maziuk wrote: > On 5/7/24 06:03, Jan Høydahl wrote: > > Hi, > > > > If you have not tuned your heap, it will default to 512m. THe lines in > solr.in.sh prefixed with hash # are comments and do nothing. > If course, it was just to show the curren

Re: Memory tuning advice for small single-node Solr instance

2024-05-07 Thread Imran Chaudhry
Hello Jan, On Tue, May 7, 2024, 12:05 PM Jan Høydahl wrote: > Hi, > > If you have not tuned your heap, it will default to 512m. THe lines in > solr.in.sh prefixed with hash # are comments and do nothing. > If you experience OOME during use that is likely not enough for your use > case. > So edit

Re: Memory tuning advice for small single-node Solr instance

2024-05-07 Thread Dmitri Maziuk
On 5/7/24 06:03, Jan Høydahl wrote: Hi, If you have not tuned your heap, it will default to 512m. THe lines in solr.in.sh prefixed with hash # are comments and do nothing. If you experience OOME during use that is likely not enough for your use case. If it's the *kernel* OOM killer: ... the

Re: Memory tuning advice for small single-node Solr instance

2024-05-07 Thread Jan Høydahl
Hi, If you have not tuned your heap, it will default to 512m. THe lines in solr.in.sh prefixed with hash # are comments and do nothing. If you experience OOME during use that is likely not enough for your use case. So edit the file, remove the hash and set SOLR_HEAP=1024m to give yourself some m

Re: Memory tuning advice for small single-node Solr instance

2024-05-06 Thread Rajani M
What type of queries do you send to Solr, what is the QTime returned per query on an average, and how many queries per second/minute? Try increasing the heap to 512 mb and 1024 mb and test it. Solr 9.6 supports QueryLimits termination which should avoid queries from excessive usage of resources and