Re: Help to determine why an optimized index is proportionaly too big.

2009-04-10 Thread Andrzej Bialecki
Chris Hostetter wrote: : The second stage index failed an optimization with a disk full exception : (I had to move it to another lucene machine with a larger disk partition : to complete the optimization. Is there a reason why a 22 day index would : be 10x the size of an 8 day index when the do

Re: Help to determine why an optimized index is proportionaly too big.

2009-04-09 Thread Koji Sekiguchi
Dan OConnor wrote: Thanks for the feed back Chris. Can you (or someone else on the list) tell me about the IndexMerge tool? Please see: http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/org/apache/lucene/misc/IndexMergeTool.html Koji -

RE: Help to determine why an optimized index is proportionaly too big.

2009-04-09 Thread Dan OConnor
determine why an optimized index is proportionaly too big. : The second stage index failed an optimization with a disk full exception : (I had to move it to another lucene machine with a larger disk partition : to complete the optimization. Is there a reason why a 22 day index would : be 10x the

Re: Help to determine why an optimized index is proportionaly too big.

2009-04-09 Thread Michael McCandless
On Thu, Apr 9, 2009 at 6:46 PM, Chris Hostetter wrote: > > : The second stage index failed an optimization with a disk full exception > : (I had to move it to another lucene machine with a larger disk partition > : to complete the optimization. Is there a reason why a 22 day index would > : be 10x

Re: Help to determine why an optimized index is proportionaly too big.

2009-04-09 Thread Chris Hostetter
: The second stage index failed an optimization with a disk full exception : (I had to move it to another lucene machine with a larger disk partition : to complete the optimization. Is there a reason why a 22 day index would : be 10x the size of an 8 day index when the document indexing rate is

Re: Help to determine why an optimized index is proportionaly too big.

2009-04-02 Thread Michael McCandless
On Wed, Apr 1, 2009 at 5:20 PM, Dan OConnor wrote: > All: > > We are using java lucene 2.3.2 to index a fairly large number of documents > (roughly 400,000 per day). We have divided the time history into various > depths. > > Our first stage covers 8 days and our next stage covers 22. The index