[
https://issues.apache.org/jira/browse/LUCENE-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141049#comment-15141049
]
Shawn Heisey edited comment on LUCENE-7020 at 2/10/16 4:02 PM:
---------------------------------------------------------------
I have no benchmark data, only personal experience, which was a number of years
ago:
With the two main settings for TMP at 35 (and no explicit setting), I saw the
total number of segments during and after a full reindex hovering between 70
and 100. An optimize on an index like this turned out to be a two phase
process -- creating a handful of very large segments and a few tiny segments,
then a second pass where those segments were merged down to a single segment.
After bumping maxMergeAtOnceExplicit to 105, an optimize completed in half the
time and only did a single merge.
was (Author: elyograg):
I have no benchmark data, only personal experience, which was a number of years
ago. I only have this personal experience:
With the two main settings for TMP at 35 (and no explicit setting), I saw the
total number of segments during and after a full reindex hovering between 70
and 100. An optimize on an index like this turned out to be a two phase
process -- creating a handful of very large segments and a few tiny segments,
then a second pass where those segments were merged down to a single segment.
After bumping maxMergeAtOnceExplicit to 105, an optimize completed in half the
time and only did a single merge.
> TieredMergePolicy - cascade maxMergeAtOnce setting to maxMergeAtOnceExplicit
> ----------------------------------------------------------------------------
>
> Key: LUCENE-7020
> URL: https://issues.apache.org/jira/browse/LUCENE-7020
> Project: Lucene - Core
> Issue Type: Improvement
> Affects Versions: 5.4.1
> Reporter: Shawn Heisey
> Assignee: Shawn Heisey
> Attachments: LUCENE-7020.patch
>
>
> SOLR-8621 covers improvements in configuring a merge policy in Solr.
> Discussions on that issue brought up the fact that if large values are
> configured for maxMergeAtOnce and segmentsPerTier, but maxMergeAtOnceExplicit
> is not changed, then doing a forceMerge is likely to not work as expected.
> When I first configured maxMergeAtOnce and segmentsPerTier to 35 in Solr, I
> saw an optimize (forceMerge) fully rewrite most of the index *twice* in order
> to achieve a single segment, because there were approximately 80 segments in
> the index before the optimize, and maxMergeAtOnceExplicit defaults to 30. On
> advice given via the solr-user mailing list, I configured
> maxMergeAtOnceExplicit to 105 and have not had that problem since.
> I propose that setting maxMergeAtOnce should also set maxMergeAtOnceExplicit
> to three times the new value -- unless the setMaxMergeAtOnceExplicit method
> has been invoked, indicating that the user wishes to set that value
> themselves.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
