Re: Reindexing leaving behind 0 live doc segments

2023-09-13 Thread Uwe Schindler
It looks like your code has a leak and does not close all IndexReaders/Writers that you use during your custom code in Solr. It is impossible to review this from outside. You shuld use the Solr provided SolrIndexWriter and SolrIndexSearcher to do your custom stuff and let Solr manage them. U

Re: Reindexing leaving behind 0 live doc segments

2023-09-09 Thread Rahul Goswami
Uwe, Thanks for the response. I have openSearcher=false in autoCommit, but I do have an autoSoftCommit interval of 5 minutes configured as well which should open a searcher. In vanilla Solr, without my code, I see that if I completely reindex all documents in a segment (via a client call), the segm

Re: Reindexing leaving behind 0 live doc segments

2023-09-04 Thread Uwe Schindler
Hi, in Solr the empty segment keeps open as long as there is a Searcher still open. At some point the empty segment (100% deletions) will be deleted, but you have to wait until SolIndexSearcher has restarted. Maybe check your solrconfig.xml and check if openSearcher is enabled after autoSoftC

Re: Reindexing leaving behind 0 live doc segments

2023-08-31 Thread Rahul Goswami
Stefan, Mike, Appreciate your responses! I spent some time analyzing your inputs and going further down the rabbit hole. Stefan, I looked at the IndexRearranger code you referenced where it tries to drop the segment. I see that it eventually gets handled via IndexFileDeleter.checkpoint() through f

Re: Reindexing leaving behind 0 live doc segments

2023-08-31 Thread Michael McCandless
Hi Rahul, Please do not pursue Approach 2 :) ReadersAndUpdates.release is not something the application should be calling. This path can only lead to pain. It sounds to me like something in Solr is holding an old reader (maybe the last commit point, or reader prior to the refresh after you re-i

Re: Reindexing leaving behind 0 live doc segments

2023-08-30 Thread Stefan Vodita
Hi Rahul, What you're describing sounds similar to index rearranging [1], although in that case the reindexing is done in a new index. The last commit in the IndexRearranger class added support for reading and reindexing deletes - maybe having a look at that and at the Javadoc would help? Stefan

Re: Reindexing leaving behind 0 live doc segments

2023-08-30 Thread Rahul Goswami
Thanks for the response Mikhail. I don't think I am looking for forceMergeDeletes() though since it could be more expensive than I would like and I only want to see the unreferenced segments with 0 live docs to be deleted. Just the way they get deleted with a commit=true option or even softDelete.

Re: Reindexing leaving behind 0 live doc segments

2023-08-28 Thread Mikhail Khludnev
Hi Rahul. Are you looking for https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/index/IndexWriter.html#forceMergeDeletes() ? On Tue, Aug 29, 2023 at 5:20 AM Rahul Goswami wrote: > Hello, > I am trying to execute a program to read documents segment-by-segment and > reindex to the same i

Re: Reindexing after database change

2012-07-28 Thread Jack Krupansky
Lucene itself doesn't "pull" data in. Rather, you need to "push" data into it. With Solr you can push data in from the command line using standard tools such as curl or Java programs or any number of scripting tools or external "push" connector systems such as Nutch or ManifoldCF, but with Luce

Re: Reindexing

2006-02-08 Thread Erik Hatcher
Sorry, I've no clue as I've never used Hibernate and thus never touched its Lucene support. Erik On Feb 8, 2006, at 1:18 AM, Raul Raja Martinez wrote: Hi Eric, I'm in the same situation, I wouldn't normally ask something related to hibernate here but I posted something similar in

Re: Reindexing

2006-02-07 Thread Raul Raja Martinez
Hi Eric, I'm in the same situation, I wouldn't normally ask something related to hibernate here but I posted something similar in the hibernate forums on Jan 16th but still haven't got any response. http://forum.hibernate.org/viewtopic.php?t=954137&highlight=lucene It is really obvious that if

Re: Reindexing

2006-02-07 Thread Amol Bhutada
Hi Revati, If I am not wrong you must be using hibernate intercepter to get the evens on value objects. You will get all lifecycle event in the class so you should handle lucene index addition/updation/deletion there. But I had faced locking problem with hibernate as multiple writers get opened

Re: Reindexing

2006-02-07 Thread Erik Hatcher
You may likely get better response by posting in the Hibernate list. Erik On Feb 7, 2006, at 7:58 AM, revati joshi wrote: Hello lucene members, i'm the silent member of this group.last week i had sent some query regarding reindexing,but i dn't