[
https://issues.apache.org/jira/browse/LUCENE-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834856#comment-16834856
]
Atri Sharma commented on LUCENE-8757:
-------------------------------------
Hi [~simonw]
bq. if the previous segment was smallish then group is non-null? I think you
should test these cases, maybe add a random test and randomize the order or the
segments?
I dont think that case is possible, since we sort LeafReaderContexts based on
the number of documents per segment in descending order. Hence, no
LeafReaderContext can be succeded by one which has more documents than its
predecessor. I agree with your thought of having a random test with variety of
configurations for segment size distributions.
bq.can and should be replaced by:
Fixed, thanks.
[^LUCENE-8757.patch]
> Better Segment To Thread Mapping Algorithm
> ------------------------------------------
>
> Key: LUCENE-8757
> URL: https://issues.apache.org/jira/browse/LUCENE-8757
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Atri Sharma
> Priority: Major
> Attachments: LUCENE-8757.patch, LUCENE-8757.patch, LUCENE-8757.patch
>
>
> The current segments to threads allocation algorithm always allocates one
> thread per segment. This is detrimental to performance in case of skew in
> segment sizes since small segments also get their dedicated thread. This can
> lead to performance degradation due to context switching overheads.
>
> A better algorithm which is cognizant of size skew would have better
> performance for realistic scenarios
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
