[
https://issues.apache.org/jira/browse/LUCENE-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-2770:
----------------------------------
Description:
This is a spin-off from LUCENE-2769:
Currently SegmentMerger has some optimizations when it merges segments that are
SegmentReaders (e.g. when doing normal indexing or optimizing). But when you do
IndexWriter.addIndexes(IndexReader...) the listed IndexReaders may not really
be per-segment. SegmentMerger should track down all passed in reads down to the
lowest level (Segment)Reader (or other atomic readers like
SlowMultiReaderWrapper) and then merge. We can then remove most MultiFields
usage (except term merging itsself) and clean up the code.
This especially saves lots of memory for merging norms, as no longer the
duplicate norms arrays are created when MultiReaders are used!
was:
This is a spin-off from LUCENE-2769:
Currently SegmentReader has some optimizations when it merges segments that are
itsself SegmentReaders (e.g. when doing normal indexing or optimizing). But
when you do IndexWriter.addIndexes(IndexReader...) the listed IndexReaders may
not really be per-segment. SegmentMerger should track down all passed in reads
down to the lowest level (Segment)Reader (or other atomic readers like
SlowMultiReaderWrapper) and then merge. We can then remove most MultiFields
usage (except term merging itsself) and clean up the code.
This especially saves lots of memory for merging norms, as no longer the
duplicate norms arrays are created when MultiReaders are used!
> Optimize SegmentMerger to work on atomic (Segment)Readers where possible
> ------------------------------------------------------------------------
>
> Key: LUCENE-2770
> URL: https://issues.apache.org/jira/browse/LUCENE-2770
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2770.patch
>
>
> This is a spin-off from LUCENE-2769:
> Currently SegmentMerger has some optimizations when it merges segments that
> are SegmentReaders (e.g. when doing normal indexing or optimizing). But when
> you do IndexWriter.addIndexes(IndexReader...) the listed IndexReaders may not
> really be per-segment. SegmentMerger should track down all passed in reads
> down to the lowest level (Segment)Reader (or other atomic readers like
> SlowMultiReaderWrapper) and then merge. We can then remove most MultiFields
> usage (except term merging itsself) and clean up the code.
> This especially saves lots of memory for merging norms, as no longer the
> duplicate norms arrays are created when MultiReaders are used!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
