Re: Strategies for Real-Time Data Updates in Solr Without Compromising Latency

2023-08-28 Thread Mikhail Khludnev
Hi, here are a few considerations. You can try to update docvalues only the price column as whole (it's worth to update as many docs as possible). https://solr.apache.org/guide/solr/latest/indexing-guide/partial-document-updates.html#in-place-updates Note: docValues only fields are searchable. This

Re: Re-index after upgrade

2023-08-28 Thread Jan Høydahl
Are you sure that 9.x refuse to open an index first created in 7.x? I thought that strict policy was only needed in 8.0 due to a particular lossy data structure change, and that 9.x is more lenient? Jan > 24. aug. 2023 kl. 22:38 skrev Shawn Heisey : > > Version 9 behaves the same as 8. If the

Re: solrj client memory leak via ThreadLocal in solrj.impl.Http2SolrClient?

2023-08-28 Thread Vincenzo D'Amore
Hi Tim, have you figured out the problem? Just curious to know what you have done at the end. On Fri, Aug 25, 2023 at 4:48 PM Vincenzo D'Amore wrote: > Just my 2 cent:, I have always used solr clients as singletons. You have > to instantiate them only once and reuse them forever. > > On Fri, 25

Re: solrj client memory leak via ThreadLocal in solrj.impl.Http2SolrClient?

2023-08-28 Thread Tim Funk
I reverted to HttpSolrClient. That seems to have plugged the leak. As for root cause, I haven't had time to dig farther. Since this happens regardless of reusing SolrClient vs instantiating a new one, I'm hoping that's a data point of interest. But as for constructing a "simple" test to reproduce,

Re: solrj client memory leak via ThreadLocal in solrj.impl.Http2SolrClient?

2023-08-28 Thread Vincenzo D'Amore
Hi Tim, thanks for letting me know, I experienced the same problem, my application became unstable and crashed. My first implementation was very similar to yours and relied heavily on try-with-resources java statements with CloudSolrClient. As said in my previous email, I ended up using the solr cl

Re: Re-index after upgrade

2023-08-28 Thread Shawn Heisey
On 8/28/23 05:03, Jan Høydahl wrote: Are you sure that 9.x refuse to open an index first created in 7.x? I thought that strict policy was only needed in 8.0 due to a particular lossy data structure change, and that 9.x is more lenient? I haven't actually tried it, but I believe N-1 is enforce

Re: Weird issue -- pulling results with cursorMark gets fewer documents than numFound

2023-08-28 Thread Chris Hostetter
: Schema meets the requirements for Atomic Update, so we are doing a migration : by querying the old cluster and writing to the new cluster. We are doing it in : batches by filtering on one of the fields, and using cursorMark to efficiently : page through the results. ... : The query thre

Re: Re-index after upgrade

2023-08-28 Thread Rahul Goswami
Yep, that check is present in Lucene 9.x as well. It will refuse to open an index created in 7.x. https://github.com/apache/lucene/blob/releases/lucene/9.4.0/lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java#L345 https://github.com/apache/lucene/blob/releases/lucene/9.4.0/lucene/core

Re: Weird issue -- pulling results with cursorMark gets fewer documents than numFound

2023-08-28 Thread Shawn Heisey
On 8/28/23 11:42, Chris Hostetter wrote: I assume you mean one of the batches always indexes 5 fewer documents then 'rows=N' param (ie: the query batch size) ... correct? You're talking about the total numFound being higher then the index count? The query uses rows=1, which is configurable

Re: Re-index after upgrade

2023-08-28 Thread Jan Høydahl
Ok, thanks for the clarification. Jan > 28. aug. 2023 kl. 20:43 skrev Rahul Goswami : > > Yep, that check is present in Lucene 9.x as well. It will refuse to open an > index created in 7.x. > > https://github.com/apache/lucene/blob/releases/lucene/9.4.0/lucene/core/src/java/org/apache/lucene/in

Registration open for Community Over Code North America

2023-08-28 Thread Rich Bowen
Hello! Registration is still open for the upcoming Community Over Code NA event in Halifax, NS! We invite you to register for the event https://communityovercode.org/registration/ Apache Committers, note that you have a special discounted rate for the conference at US$250. To take advantage of th