Re: Slowness in Solr Optimize

2022-12-13 Thread David Hastings
Interesting, actually one method I used was to use Memcached to get the next page of results stored in the web servers memory, worked pretty slick if you had enough memory On Tue, Dec 13, 2022 at 3:01 PM Walter Underwood wrote: > Sticky sessions only help for later pages. They don’t direct new q

Re: Slowness in Solr Optimize

2022-12-13 Thread Walter Underwood
Sticky sessions only help for later pages. They don’t direct new queries to servers that already have that query cached. That is a bigger win than second pages. I’ve thought about making a hash of the query-meaningful params (q, bq, …) and routing based on that, but that has other problems. Whe

Re: Slowness in Solr Optimize

2022-12-13 Thread David Hastings
Ah, that makes sense. If you can do sticky sessions and such with your balancers, plus I never had to deal with the throughput of something like Netflix, so for mine and most use cases, I still feel one very hot server is better than N warm ones. "but AWS load balancers aren’t very smart." - agre

Re: Slowness in Solr Optimize

2022-12-13 Thread Walter Underwood
I’ve always run load balancers, starting with Solr 1.2 at Netflix. Failover (cold) spares have cold caches, so have slow performance until the cache fills. I configure N+1 capacity, where N servers can handle the expected load, then we add one for failure handling. All the spares are hot. I eve

Re: Slowness in Solr Optimize

2022-12-13 Thread Dave
Sounds like you should contact aws about it since it’s not a solr issue if the qtimes haven’t increased in the solr logs. And again, don’t load balance but that’s my personal opinion > On Dec 13, 2022, at 6:50 AM, Pradeep wrote: > > Hi, > > I cant change it to NLB at this moment, firstly w

Re: Slowness in Solr Optimize

2022-12-13 Thread Dave
Ha I meant qtimes not atone. Also in general you shouldn’t use a load balancer with solr, since you won’t be able to keep the index hot and n memory for each subsequent query if you are paging through results. The best way in my experience is to have failovers for your nodes, instead of load ba

Re: Slowness in Solr Optimize

2022-12-13 Thread Pradeep
Hi, I cant change it to NLB at this moment, firstly why timeout we need to understand no clue at this moment. It works fine if i increase timeout from 60 secs to 4000 secs. Its same code working in classic load balancer but with ALB we have this issue. Thanks, Pradeep On Tue, 13 Dec, 2022, 11:0

Re: Slowness in Solr Optimize

2022-12-13 Thread Deepak Goel
Can you try 'Network Load Balancer" in aws? Deepak "The greatness of a nation can be judged by the way its animals are treated - Mahatma Gandhi" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In

Re: Slowness in Solr Optimize

2022-12-12 Thread Shawn Heisey
On 12/12/22 13:14, Pradeep wrote: How to check time taken by solr to execute api? Also can you share me solr doc how we can query manually solr index for specific record or any api's you can provide. Not entirely sure what you are asking here.  I will try to answer what I can discern. In sol

Re: Slowness in Solr Optimize

2022-12-12 Thread Dave
You can check the atones to see if solr itself actually slowed down. As solr has nothing to do with a load balancer I doubt it has. Also you used a sentence that concerns me, clearing out the deleted documents, which sounds like an optimize command. You as a user should never use that, let sol

Re: Slowness in Solr Optimize

2022-12-12 Thread Pradeep
Thank you shawn. I will have look, I have checked the solr.log file but didn't see any error, after load balancer changes we are seeing this issue. How to check time taken by solr to execute api? Also can you share me solr doc how we can query manually solr index for specific record or any api's y

Re: Slowness in Solr Optimize

2022-12-12 Thread Shawn Heisey
On 12/12/22 08:14, Pradeep wrote: We have migrated Apache solr aws load balancer from classic to Application load balancer. We havent changed anything in solr other than migrating load balancer but solr taking more time while committing in Solr. Fyi, we are processing 200 records every batch, t