On Thu, Jul 7, 2022 at 3:31 AM Christopher Schultz < ch...@christopherschultz.net> wrote:
> Deepak, > > On 7/5/22 09:30, Deepak Goel wrote: > > Do you mind telling us which java version are you using? > > It doesn't matter. Xms == Xmx is always a good idea for server > processes. For running Keystore Explorer, let the JVM manage the heap > size fluctuations. For servers, tell the JVM what you want. > Chris, I am asking for the java version because there are many more parameters to tune the GC other than (Xms == Xmx) > -chris > > > On Tue, 5 Jul 2022, 17:55 Dave, <hastings.recurs...@gmail.com> wrote: > > > >> Also a good rule of thumb I found is set your xmx and xms, maximum and > >> minimum memory for the heap to be exactly the same, you don’t want Java > to > >> try to figure it out, > >> > >>> On Jul 5, 2022, at 7:52 AM, Ritvik Sharma <ritvik.s...@gmail.com> > wrote: > >>> > >>> Just Add java memory parameters in solr config which should not be > more > >>> than 75% of total RAM. and use G1GC. > >>> > >>> > >>>> On Tue, 5 Jul 2022 at 4:59 PM, Deepak Goel <deic...@gmail.com> wrote: > >>>> > >>>> I would also suggest you look at which GC mechanism you use. > Increasing > >> RAM > >>>> and Heap-Size might result in the application freezed for a long time > >>>> (during GC). > >>>> > >>>> Deepak > >>>> "The greatness of a nation can be judged by the way its animals are > >> treated > >>>> - Mahatma Gandhi" > >>>> > >>>> +91 73500 12833 > >>>> deic...@gmail.com > >>>> > >>>> Facebook: https://www.facebook.com/deicool > >>>> LinkedIn: www.linkedin.com/in/deicool > >>>> > >>>> "Plant a Tree, Go Green" > >>>> > >>>> Make In India : http://www.makeinindia.com/home > >>>> > >>>> > >>>>> On Tue, Jul 5, 2022 at 4:43 PM Dave <hastings.recurs...@gmail.com> > >> wrote: > >>>>> > >>>>> Exactly. You could have the best engineer on your continent but the > end > >>>>> result is the same, more metal. I could build a very fast search > >> server > >>>>> for less than a week of my salary so what’s the point of wasting two > >>>> weeks > >>>>> trying to solve a problem when the solution is literally just right > >>>> there, > >>>>> a big ssd and a lot of memory, then I could work on things that are > >>>>> actually important rather than try to squeeze blood from a turnip. > >>>>> > >>>>>> On Jul 5, 2022, at 6:11 AM, Charlie Hull < > >>>>> ch...@opensourceconnections.com> wrote: > >>>>>> > >>>>>> I think you're missing my point. > >>>>>> > >>>>>> Good engineers, even mediocre ones, are expensive, and the great > ones > >>>>> are rare. It's a tedious task chasing tiny performance gains when you > >>>> know > >>>>> you're limited by the hardware and a bored engineer might just go and > >>>> look > >>>>> for another job. So if you fail to realise that a capital expense for > >>>>> hardware or hosting is necessary, you run the risk of losing the > people > >>>>> that make your search engine work (even if they stay they could also > be > >>>>> doing something more useful to your business). > >>>>>> > >>>>>> Charlie > >>>>>> > >>>>>>> On 05/07/2022 10:49, Deepak Goel wrote: > >>>>>>> If you are tearing your hair out on 'Number of Hours' required for > >>>>> tuning > >>>>>>> your software, it's time you switch to a better quality performance > >>>>>>> engineer. > >>>>>>> > >>>>>>> Deepak > >>>>>>> "The greatness of a nation can be judged by the way its animals are > >>>>> treated > >>>>>>> - Mahatma Gandhi" > >>>>>>> > >>>>>>> +91 73500 12833 > >>>>>>> deic...@gmail.com > >>>>>>> > >>>>>>> Facebook:https://www.facebook.com/deicool > >>>>>>> LinkedIn:www.linkedin.com/in/deicool > >>>>>>> > >>>>>>> "Plant a Tree, Go Green" > >>>>>>> > >>>>>>> Make In India :http://www.makeinindia.com/home > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull< > >>>>> ch...@opensourceconnections.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Equally it's not a good management practice to burn engineering > >> hours > >>>>>>>> trying to optimise performance to avoid spending (often much less) > >>>>> money > >>>>>>>> on sufficient hardware to do the job. I've seen this happen many > >>>> times, > >>>>>>>> sadly. > >>>>>>>> > >>>>>>>> Charlie > >>>>>>>> > >>>>>>>> On 05/07/2022 10:33, Deepak Goel wrote: > >>>>>>>>> Not a good software engineering practice to beef up the hardware > >>>>> blindly. > >>>>>>>>> Of Course when you have tuned the software to a point where you > >>>> can't > >>>>>>>> tune > >>>>>>>>> anymore, you can then turn your eyes to hardware. > >>>>>>>>> > >>>>>>>>> Deepak > >>>>>>>>> "The greatness of a nation can be judged by the way its animals > are > >>>>>>>> treated > >>>>>>>>> - Mahatma Gandhi" > >>>>>>>>> > >>>>>>>>> +91 73500 12833 > >>>>>>>>> deic...@gmail.com > >>>>>>>>> > >>>>>>>>> Facebook:https://www.facebook.com/deicool > >>>>>>>>> LinkedIn:www.linkedin.com/in/deicool > >>>>>>>>> > >>>>>>>>> "Plant a Tree, Go Green" > >>>>>>>>> > >>>>>>>>> Make In India :http://www.makeinindia.com/home > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<hastings.recurs...@gmail.com > > > >>>>>>>> wrote: > >>>>>>>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which > helps a > >>>>> lot. > >>>>>>>> It > >>>>>>>>>> comes to a point where money on hardware will outweigh money on > >>>>>>>> engineering > >>>>>>>>>> man power hours, and still come to the same conclusion. As much > >> ram > >>>>> as > >>>>>>>> your > >>>>>>>>>> rack can take and as big and fast of a raid ssd drive it can > take. > >>>>>>>> Remember > >>>>>>>>>> since solr is always meant to be destroyed and recreated you > don’t > >>>>> have > >>>>>>>> to > >>>>>>>>>> worry much about hardware failure if you just buy two of > >> everything > >>>>> and > >>>>>>>>>> have a backup server ready and waiting to take over while the > >>>>> original > >>>>>>>>>> fails and is reconstructed. > >>>>>>>>>> > >>>>>>>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<apa...@elyograg.org> > >>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> On 7/4/22 03:01, Mike wrote: > >>>>>>>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. > Solr > >>>>> eats > >>>>>>>> up > >>>>>>>>>> all > >>>>>>>>>>>> the memory and because of that PHP works very, very slowly. > What > >>>>> can I > >>>>>>>>>> do? > >>>>>>>>>>> Solr is a Java program. A Java program will never directly use > >>>> more > >>>>>>>>>> memory than you specify for the max heap size. We cannot make > any > >>>>>>>> general > >>>>>>>>>> recommendations about what heap size you need, because there is > a > >>>>> good > >>>>>>>>>> chance that any recommendation we make would be completely wrong > >>>> for > >>>>>>>> your > >>>>>>>>>> install. I did see that someone recommended not going above 31G > >>>> ... > >>>>> and > >>>>>>>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers > >>>>>>>> instead of > >>>>>>>>>> 32-bit. So a heap size of 32 GB actually has LESS memory > >> available > >>>>>>>> than a > >>>>>>>>>> heap size of 31 GB. > >>>>>>>>>>> The OS will use additional memory beyond the heap for caching > the > >>>>> index > >>>>>>>>>> data, but that is completely outside of Solr's control. Note > that > >>>>> 64GB > >>>>>>>>>> total memory for a 500GB index is almost certainly not enough > >>>> memory, > >>>>>>>>>> ESPECIALLY if the same server is used for things other than > Solr. > >>>> I > >>>>>>>> wrote > >>>>>>>>>> the following wiki page: > >>>>>>>> > >>>>> > >> > https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems > >>>>>>>>>>> Others have recommended that you run Solr on dedicated hardware > >>>>> that is > >>>>>>>>>> not used for any other purpose. I concur with that > >> recommendation. > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Shawn > >>>>>>>>>>> > >>>>>>>> -- > >>>>>>>> Charlie Hull - Managing Consultant at OpenSource Connections > Limited > >>>>>>>> Founding member of The Search Network< > >>>> http://www.thesearchnetwork.com> > >>>>>>>> and co-author of Searching the Enterprise > >>>>>>>> < > >>>>>>>> > >>>>> > >>>> > >> > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > >>>>>>>> tel/fax: +44 (0)8700 118334 > >>>>>>>> mobile: +44 (0)7767 825828 > >>>>>>>> > >>>>>>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 > >> Berlin > >>>>>>>> Amtsgericht Charlottenburg | HRB 230712 B > >>>>>>>> Geschäftsführer: John M. Woodell | David E. Pugh > >>>>>>>> Finanzamt: Berlin Finanzamt für Körperschaften II > >>>>>>>> > >>>>>>>> -- > >>>>>>>> This email has been checked for viruses by AVG. > >>>>>>>> https://www.avg.com > >>>>>>>> > >>>>>> -- > >>>>>> Charlie Hull - Managing Consultant at OpenSource Connections Limited > >>>>>> Founding member of The Search Network < > >> http://www.thesearchnetwork.com > >>>>> > >>>>> and co-author of Searching the Enterprise < > >>>>> > >>>> > >> > https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf > >>>>>> > >>>>>> tel/fax: +44 (0)8700 118334 > >>>>>> mobile: +44 (0)7767 825828 > >>>>>> > >>>>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 > Berlin > >>>>>> Amtsgericht Charlottenburg | HRB 230712 B > >>>>>> Geschäftsführer: John M. Woodell | David E. Pugh > >>>>>> Finanzamt: Berlin Finanzamt für Körperschaften II > >>>>>> > >>>>>> -- > >>>>>> This email has been checked for viruses by AVG. > >>>>>> https://www.avg.com > >>>>> > >>>> > >> > > >
