m/ -- Solr - Lucene - Nutch
>
>
>
>
>
> From: Jason Rutherglen
> To: java-user@lucene.apache.org
> Sent: Wed, January 13, 2010 5:54:38 PM
> Subject: Re: Max Segmentation Size when Optimizing Index
>
> Yes... You could hack LogMergePolicy to do something else.
__
> From: Jason Rutherglen
> To: java-user@lucene.apache.org
> Sent: Wed, January 13, 2010 5:54:38 PM
> Subject: Re: Max Segmentation Size when Optimizing Index
>
> Yes... You could hack LogMergePolicy to do something else.
>
> I use optimise(numse
I think Jason meant "15-20GB segments"?
Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
From: Jason Rutherglen
To: java-user@lucene.apache.org
Sent: Wed, January 13, 2010 5:54:38 PM
Subject: Re: Max Segmentation Size when Optimi
Yes... You could hack LogMergePolicy to do something else.
I use optimise(numsegments:5) regularly on 80GB indexes, that if
optimized to 1 segment, would thrash the IO excessively. This works
fine because 15-20GB indexes are plenty large and fast.
On Wed, Jan 13, 2010 at 2:44 PM, Trin Chavalittu
Seems like optimize() only cares about final number of segments rather than
the size of the segment. Is it so?
On Wed, Jan 13, 2010 at 2:35 PM, Jason Rutherglen <
jason.rutherg...@gmail.com> wrote:
> There's a different method in LogMergePolicy that performs the
> optimize... Right, so normal mer
There's a different method in LogMergePolicy that performs the
optimize... Right, so normal merging uses the findMerges method, then
there's a findMergeOptimize (method names could be inaccurate).
On Wed, Jan 13, 2010 at 2:29 PM, Trin Chavalittumrong wrote:
> Do you mean MergePolicy is only used
Do you mean MergePolicy is only used during index time and will be ignored
by by the Optimize() process?
On Wed, Jan 13, 2010 at 1:57 PM, Jason Rutherglen <
jason.rutherg...@gmail.com> wrote:
> Oh ok, you're asking about optimizing... I think that's a different
> algorithm inside LogMergePolicy.
Oh ok, you're asking about optimizing... I think that's a different
algorithm inside LogMergePolicy. I think it ignores the maxMergeMB
param.
On Wed, Jan 13, 2010 at 1:49 PM, Trin Chavalittumrong wrote:
> Thanks, Jason.
>
> Is my understanding correct that LogByteSizeMergePolicy.setMaxMergeMB(10
Thanks, Jason.
Is my understanding correct that LogByteSizeMergePolicy.setMaxMergeMB(100)
will prevent
merging of two segments that is larger than 100 Mb each at the optimizing
time?
If so, why do think would I still see segment that is larger than 200 MB?
On Wed, Jan 13, 2010 at 1:43 PM, Jaso
Hi Trin,
There was recently a discussion about this, the max size is
for the before merge segments, rather than the resultant merged
segment (if that makes sense). It'd be great if we had a merge
policy that limited the resultant merged segment, though that'd
by a rough approximation at best.
Jas
10 matches
Mail list logo
