Re: Lucene multithreaded indexing problems

2013-11-23 Thread Daniel Penning
G1 and CMS are both tuned primarily for low pauses which is typically prefered for searching an index. In this case i guess that indexing throughput is prefered in which case using ParallelGC might be the better choice. Am 23.11.2013 17:15, schrieb Uwe Schindler: Hi, Maybe your heap size is

Re: Lucene multithreaded indexing problems

2013-11-23 Thread Daniel Penning
Maybe you should turn on Garbage Collection logging to confirm that you are running into some kind of memory problem. (start JVM with -verbose:gc) If the GC is running very often as soon as your indexing process slows down, i would suggest you to create a heapdump and check what the memory is us

Re: Taking backup of a Lucene index

2013-06-06 Thread Daniel Penning
I do my backups by creating a new index at the backup target and copying everything over with IndexWriter#addIndexes(IndexReader... readers). In the future i am also planing on using a RateLimitedDirectoryWrapper to reduce the influence of the running backup on the rest of the system. Am 06.06

Re: Lucene 3.6.2 deleteDocument(docNum) and undeleteAll

2013-05-24 Thread Daniel Penning
The changes made by a IndexWriter since the last commit() will be undone if you close it with IndexWriter#rollback(). Afaik using IndexReader#undeleteAll to undo previously done deletes is a very bad way as it will not only undelete the documents deleted by the current IndexReader but all dele

Re: updating/deleting uncomitted documents

2013-05-06 Thread Daniel Penning
I'm sorry, there was a bug in my test case which caused deleted documents to be shown as live documents. Am 06.05.2013 13:36, schrieb Michael McCandless: On Mon, May 6, 2013 at 7:32 AM, Daniel Penning wrote: If a Document is updated multiple times in one transaction it may end up wit

updating/deleting uncomitted documents

2013-05-06 Thread Daniel Penning
Hi If a Document is updated multiple times in one transaction it may end up with old versions not getting deleted, depending on when the IndexWriter got flushed. I couln't find an details about this behaviour in the documentation. Is this the expected behaviour and will forcing a flush by ope