Slowness in Solr Optimize
Hi Team, We have migrated Apache solr aws load balancer from classic to Application load balancer. We havent changed anything in solr other than migrating load balancer but solr taking more time while committing in Solr. Fyi, we are processing 200 records every batch, this is failing at last stage of commit in solr (deleting stale records in solr). But same logic working fine from many years in classic load balancer but after migration to ALB this is happenning. Regards, Pradeep
Re: Slowness in Solr Optimize
Thank you shawn. I will have look, I have checked the solr.log file but didn't see any error, after load balancer changes we are seeing this issue. How to check time taken by solr to execute api? Also can you share me solr doc how we can query manually solr index for specific record or any api's you can provide. Thanks On Mon, 12 Dec, 2022, 18:51 Shawn Heisey, wrote: > On 12/12/22 08:14, Pradeep wrote: > > We have migrated Apache solr aws load balancer from classic to > Application > > load balancer. We havent changed anything in solr other than migrating > load > > balancer but solr taking more time while committing in Solr. > > > > Fyi, we are processing 200 records every batch, this is failing at last > > stage of commit in solr (deleting stale records in solr). But same logic > > working fine from many years in classic load balancer but after migration > > to ALB this is happenning. > > The message subject mentions optimize but the message body does not. > > Is optimize involved? An optimize of a large index is always going to > be slow. > > Solr does not have any kind of "deleting stale records" that I have ever > heard of, so that must be something your code, which is more difficult > for us to troubleshoot because we are not going to have any idea how to > get that code to give useful logs without a lot more information about > that code. > > If you're encountering an error from Solr then solr.log should have at > least some info about what happened and that might be enough info to > figure out the problem. > > If changing the load balancer caused issues, then you may need to talk > to someone who understands how to troubleshoot the LB. > > Basically we need a lot more information about what's happening to have > any hope of helping you. > > Thanks, > Shawn > >
Re: Slowness in Solr Optimize
Hi, I cant change it to NLB at this moment, firstly why timeout we need to understand no clue at this moment. It works fine if i increase timeout from 60 secs to 4000 secs. Its same code working in classic load balancer but with ALB we have this issue. Thanks, Pradeep On Tue, 13 Dec, 2022, 11:08 Deepak Goel, wrote: > Can you try 'Network Load Balancer" in aws? > > > Deepak > "The greatness of a nation can be judged by the way its animals are treated > - Mahatma Gandhi" > > +91 73500 12833 > deic...@gmail.com > > Facebook: https://www.facebook.com/deicool > LinkedIn: www.linkedin.com/in/deicool > > "Plant a Tree, Go Green" > > Make In India : http://www.makeinindia.com/home > > > On Tue, Dec 13, 2022 at 12:21 AM Shawn Heisey > wrote: > > > On 12/12/22 08:14, Pradeep wrote: > > > We have migrated Apache solr aws load balancer from classic to > > Application > > > load balancer. We havent changed anything in solr other than migrating > > load > > > balancer but solr taking more time while committing in Solr. > > > > > > Fyi, we are processing 200 records every batch, this is failing at last > > > stage of commit in solr (deleting stale records in solr). But same > logic > > > working fine from many years in classic load balancer but after > migration > > > to ALB this is happenning. > > > > The message subject mentions optimize but the message body does not. > > > > Is optimize involved? An optimize of a large index is always going to > > be slow. > > > > Solr does not have any kind of "deleting stale records" that I have ever > > heard of, so that must be something your code, which is more difficult > > for us to troubleshoot because we are not going to have any idea how to > > get that code to give useful logs without a lot more information about > > that code. > > > > If you're encountering an error from Solr then solr.log should have at > > least some info about what happened and that might be enough info to > > figure out the problem. > > > > If changing the load balancer caused issues, then you may need to talk > > to someone who understands how to troubleshoot the LB. > > > > Basically we need a lot more information about what's happening to have > > any hope of helping you. > > > > Thanks, > > Shawn > > > > >
Re: Solr eats up all the memory
Can you please inscribe me from this mailing list? On Mon, Jul 4, 2022 at 7:28 PM David Hastings wrote: > in my experience, yes, solr should have its own hardware, and be allowed to > eat all of it. never give it more than 31gb of jvm heap, and give it as > much memory as possible. 64GB should work fine but I can just go on amazon > and buy another 128GB for less than $500, the more the better, less than > $1000 will get you into the happy place of over 250gb, > > On Mon, Jul 4, 2022 at 5:55 AM Deepak Goel wrote: > > > I wonder if this would be helpful: > > > > https://solr.apache.org/guide/6_6/indexconfig-in-solrconfig.html > > > > https://solr.apache.org/guide/6_6/jvm-settings.html > > > > > > Deepak > > "The greatness of a nation can be judged by the way its animals are > treated > > - Mahatma Gandhi" > > > > +91 73500 12833 > > deic...@gmail.com > > > > Facebook: https://www.facebook.com/deicool > > LinkedIn: www.linkedin.com/in/deicool > > > > "Plant a Tree, Go Green" > > > > Make In India : http://www.makeinindia.com/home > > > > > > On Mon, Jul 4, 2022 at 3:18 PM Thomas Corthals > > wrote: > > > > > Hello Mike, > > > > > > If possible, run Solr on a separate machine. You're still going to need > > to > > > spec it out and configure it to your needs, but at least your client > code > > > will keep running. > > > > > > Thomas > > > > > > Op ma 4 jul. 2022 11:01 schreef Mike : > > > > > > > Hello! > > > > > > > > My Solr index size is around 500GB and I have 64GB of RAM. Solr eats > up > > > all > > > > the memory and because of that PHP works very, very slowly. What can > I > > > do? > > > > > > > > Thanks > > > > > > > > Mike > > > > > > > > > >
