> OK. I'll double check the reports. The issue I'm tackling in this app is that multiple servers are applying equal-size volumes of updates in a distributed transaction and one means of node failure detection is based on detecting a server's failure to prepare a batch of updates in a timescale proportional to other nodes. Due to recent timeout failures after upgrading Lucene, the suggestion was that the way merge operations is handled changed from 2.4 to 2.9 and now some servers took longer to apply updates and reach prepareToCommit() stage based on some shift to foreground rather than background merges in some stages.That looks not to be the case.
The theory I'm working on now is that a call to IW.updateDocuments() is blocking on a server because ConcurrentMergeScheduler has exhausted maxMergeCount due to some ongoing large segment merges.It looks like this blocking logic is not new between 2.4 and 2.9 so maybe this has just appeared now simply as a result of the index reaching a particular size/state. Cheers Mark ----- Original Message ---- From: Michael McCandless <luc...@mikemccandless.com> To: java-user@lucene.apache.org Sent: Wed, 6 October, 2010 0:18:20 Subject: Re: Merge and commit behaviour - changed between 2.4 and 2.9? Well, the merges continue running and altering the segments files, but the changes done by the merges are not committed until the app calls commit (or closes the IW). Ie, no new segments_N file is written due to merging after the last commit. Mike On Tue, Oct 5, 2010 at 6:45 PM, Mark Harwood <markharw...@yahoo.co.uk> wrote: > OK. I'll double check the reports. > So presumably when merges occur outside of transaction control (post commit) >the post-merge update of the segments_N file is managed safely somehow? > I can see the logic in continuing file changes outside of transaction control >because the document updates are safely committed (flushed as new segments) >and >the merges are essentially just a background tidying of committed state that >can >fail with no detrimental effect on the previous transaction. > > > On 5 Oct 2010, at 23:31, Michael McCandless wrote: > >> Hmm that shouldn't be the case, I think? >> >> In both 2.4 and 2.9.x (and all later versions), neither .prepareCommit >> nor .commit wait for merges. >> >> That said, if a merge happens to complete before you call those >> methods, then it is in fact committed. >> >> Mike >> >> On Tue, Oct 5, 2010 at 1:13 PM, Mark Harwood <markharw...@yahoo.co.uk> wrote: >>> Having upgraded a live system from 2.4 to 2.9.3 the client is reporting a >>>change in merge behaviour that is causing some issues with their update >>>monitoring logic. >>> >>> The suggestion is that any merge operations now complete as part of the >>>IW.prepareCommit() call rather than previously when they ran in the >>>background >>>after IW.commit calls. >>> This seems to make sense to me but I couldn't see any direct reference to >>> this >>>change in behaviour in changes.txt. >>> >>> Can anyone confirm this change between versions? >>> >>> Cheers, >>> Mark >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
