Re: NRT indexing and ControlledRealTimeReopenThread

2014-02-20 Thread Hans Lund
I've created https://issues.apache.org/jira/browse/LUCENE-5461, and attached a small test that shows the error it a setup similar to what I would like to run The 1% is a overestimation - it seems to be related to concurrent commit on the index writer Hans Lund On Thu, Feb 20, 2014 at 2:04 PM, M

Re: NRT indexing and ControlledRealTimeReopenThread

2014-02-20 Thread Michael McCandless
On Thu, Feb 20, 2014 at 7:52 AM, Hans Lund wrote: > Ok, thats also what I expected, but not what I observed ;-) Ahh, not good. > For the very huge majority of index updates reopens are not an issue, > minutes will be very fine. A very few updates are done 'interactively' and > must be in RT (or

Re: NRT indexing and ControlledRealTimeReopenThread

2014-02-20 Thread Hans Lund
Ok, thats also what I expected, but not what I observed ;-) For the very huge majority of index updates reopens are not an issue, minutes will be very fine. A very few updates are done 'interactively' and must be in RT (or as close as possible). I don't know if this is a rare use case - but we do

Re: NRT indexing and ControlledRealTimeReopenThread

2014-02-20 Thread Michael McCandless
It is intended that there are two different stale times. When a specific generation is requested, we wait for the minStaleSec since the last reopen; this is to prevent too-frequent reopens when specific gens are requested. The maxStaleSec is how long we wait between reopens for the "normal" perio