Solr Admin Page Metrics

Hey Solr community. I started playing around with the 8.8.1 docker image today and noticed issues with the JVM and host memory 'Bar' graphs from the dashboard page of the Solr Admin interface. It also appeares the "JVM" parameters were not listed here but definitely configured as expected as the

RE: zk upconfig does not recognize ZK_HOST style url

Hi Jan, Opened SOLR-15231 Thanks, Subhajit From: Jan Høydahl Sent: 09 March 2021 05:05 PM To: users@solr.apache.org Subject: Re: zk upconfig does not recognize ZK_HOST style url This so

Distributed group query, if in ` fl ` rename the unique key field name, will be NullPointerException.

Hi: Distributed group query, if in ` fl ` rename the unique key field name, will be java.lang.NullPointerException. Because of without StoredFieldsShardResponseProcessor processing rename. Can I create an issue to fix it? Sincerely yours Dawn

Re: Solr not distributing search requests among replicas

: >> 2) a single "extra" solr node in the cluster can be used as a "self : configuring" load balancer : : I’ve thought about this a bunch before, are there mechanisms to instruct : Solr to not host shards for this purpose? Maybe it deserves its own : discussion. Rules based replica placement can

Re: Increase in response time in case of collapse queries.

@florin Great advice. Null key for unique documents is really helpful. Any other such tricks that you are using to improve collapse performance ? On Tue, Mar 9, 2021, 2:45 PM Parshant Kumar wrote: > Hi Joel, > > 1) What are the response times for both methods. Saying one is faster is > not spec

Best throttling / push-back strategy for updates?

Hi, When sending updates to Solr, you often need to run multi threaded to utilize the CPU on the solr side. But how can the client (whether it is pure HTTP POST or SolrJ know whether Solr is happy with the indexing speed or not? I'm thinking of a feedback mechanism where Solr can check its load

Re: Best throttling / push-back strategy for updates?

In a master/slave system, it is OK to run as fast as possible to the master. In a cloud system, we want to keep the indexing load at a level that doesn’t interfere with queries. I do this by matching the number of indexing threads to the number of CPUs. Very roughly, two threads will keep one CPU

Re: Increase in response time in case of collapse queries.

So here are the response times: Response time for the grouped method is 167 ms for 0.65 million requests. Response time for the collapsed method is 177 ms for 0.65 million requests. If group.ngroups is used with 6+ million cardinality, then the result set size before grouping must have been small

Re: Solr Admin Page Metrics

I dug into this a l little deeper and it looks like some of the metrics reported from the Metrics API have changed between Solr 7 and 8. The main culprits seem to be os.totalPhysicalMemorySize not being calucated in Solr 8 and two missing metrics os.totalSwapSpaceSize and os.freeSwapSpaceSize w

Re: Solr Admin Page Metrics

I’d love to see a Jira issue created and a PR opened against https://github.com/apache/solr for this. Tag me and I’ll review it. > On Mar 11, 2021, at 6:13 PM, Dwane Hall wrote: > > I dug into this a l little deeper and it looks like some of the metrics > rep

Re: Solr Admin Page Metrics

Hi Eric I've raised a jira ticket (https://issues.apache.org/jira/browse/SOLR-15251) for the topic mentioned in this thread. I was unable to assign it to you or find your username to tag you in the ticket. Thanks, Dwane From: Eric Pugh Sent: Friday, 12 March

Re: Solr Admin Page Metrics

I assigned it to me (on Jira I am a David), and I’d love to see your PR. Let me know on the ticket if you need some help! > On Mar 11, 2021, at 7:04 PM, Dwane Hall wrote: > > Hi Eric I've raised a jira ticket > (https://issues.apache.org/jira/browse/SOLR-15251 >

Re: Best throttling / push-back strategy for updates?

Yes, that is what I'm recommending customers right now, to manually match indexing threads with CPUs, and that is the "manual" way. My question was rather whether we have or want to add some dynamic backoff system so that clients can just go full speed until told to back off, and thus adjust pe

Re: Best throttling / push-back strategy for updates?

The new circuit breakers might be able to offer some rate limiting. On Thu, Mar 11, 2021 at 6:25 PM Jan Høydahl wrote: > Yes, that is what I'm recommending customers right now, to manually match > indexing threads with CPUs, and that is the "manual" way. > > My question was rather whether we hav

Re: Best throttling / push-back strategy for updates?

Circuit breakers only cancel searches. They don’t touch updates. I was in that code a few weeks ago and have a patch waiting for approval. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 11, 2021, at 4:33 PM, Mike Drob wrote: > > The new circuit

Re: Best throttling / push-back strategy for updates?

I really like the idea. I too have had instances in the past where (some) updates fail because of long (ish) gc pause times due to overloading and having the option to pause indexing and give Solr a chance to catch up would very useful. I typically have a retry clause managing these issues but