OK I opened https://issues.apache.org/jira/browse/LUCENE-1611.
Christiaan, could you try out that patch to see if it fixes the
semi-infinite merging? Thanks. (You'll need to back-port to 2.4.1,
but it's a very small patch so hopefully not a problem).
Mike
On Fri, Apr 24, 2009 at 5:11 PM, Micha
On Fri, Apr 24, 2009 at 5:02 PM, Christiaan Fluit
wrote:
> Rollback does not work for me, as my IW is in auto-commit mode. It gives an
> IllegalStateException when I invoke it.
>
> A workaround that does work for me is to close and reopen the IndexWriter
> immediately after an OOME occurs.
Ahh,
Michael McCandless wrote:
- even though the commitMerge returns false, it should probably not get into
an infinite loop. Is this an internal Lucene problem or is there something I
can/should do about it myself?
Yes, something is wrong with Lucene's handling of OOME. It certainly
should not lea
On Tue, Apr 21, 2009 at 6:40 PM, Christiaan Fluit
wrote:
> I may be on to something already.
>
> I just looked at the commitMerge code and was surprised to see that the
> commitMerge message that is almost at the beginning wasn't printed. Then I
> saw the "if (hitOOM) return false;" part that tak
Christiaan Fluit wrote:
It seems that it gets up to the point to commit, but the "IW:
commitMerge done" message is never reached.
Furthermore, no exceptions are printed to the output, so
handleMergeException does not seem to have been invoked.
Should I add more debug statements elsewhere?
Michael McCandless wrote:
One question: are you using IndexWriter.close(false)? I wonder if
there's some path whereby the merges fail to abort (and simply keep
retrying) if you do that...
No, I don't.
More inlined below...
On Thu, Apr 16, 2009 at 5:42 AM, Christiaan Fluit
wrote:
I spent a
One question: are you using IndexWriter.close(false)? I wonder if
there's some path whereby the merges fail to abort (and simply keep
retrying) if you do that...
More inlined below...
On Thu, Apr 16, 2009 at 5:42 AM, Christiaan Fluit
wrote:
> I spent a lot of time on getting the stacktraces, bu
I spent a lot of time on getting the stacktraces, but JET seems to make
this impossible. Ctrl-Break, connecting with JConsole, even a "Dump
Threads" button in my UI that uses Threads.getAllStacktraces were not
able to produce a dump of all threads.
I just got an additional confirmation that th
> On Unix, one can "kill -SIGQUIT" to get a thread stack trace dump for
> all threads; do you know how to do this on Windows? If so, can you do
> that at the end when IW starts doing this infinite merging? That
> would be very helpful towards understanding why this recursion is
> happening (thoug
Hmmm, very very odd.
First off, your "1 deleted docID" is because one document hit an
exception during indexing, likely in enumerating tokens from the
TokenStream; I see this line:
IW 0 [CrawlThread]: hit exception adding document
But I think that's fine (certainly should not cause what you ar
10 matches
Mail list logo
