[
https://issues.apache.org/jira/browse/SOLR-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Shanin updated SOLR-18541:
-------------------------------
Description:
🤖 _AI text below_ 🤖 _(reviewed by Nick Shanin)_
CombinedQueryComponent.mergeIds gives wrong numFound, approximateTotalHits and
numFoundExact when a shard is partial or errors.
Three defects in the merged totals:
1. A partial shard's totals are added before its skip is tested. Its hits are
counted but its documents are dropped. With a score-only sort, a partial shard
with no per-query entry throws a NullPointerException.
2. The approximateTotalHits fallback, for a shard with no header, adds the
running total instead of that shard's own count.
3. The numFoundExact fold was inverted. SOLR-18550 (PR #5172, merged
2026-10-23) fixed this on main, and it also set the rules for numFound and
numFoundExact with several sub-queries.
The fix is PR #5117, rebased on main. It tests the skip before any totals are
added, uses the shard's own count in the fallback, and clears numFoundExact for
skipped and errored shards.
Still open: QueryComponent.mergeIds has the same leak as defect 1. A real
timeAllowed exit can leave a shorter per-query list, and the unchecked access
in CombinedQueryComponent is not changed by the PR.
was:
🤖 _AI text below_ 🤖 _(posted on behalf of Nick Shanin)_
Combined queries merge their per-query shard results in
{{CombinedQueryComponent.mergeIds}}, and three defects sit in the totals that
merge reports. First, a partial shard that sends a per-query entry is counted
in that query's totals before the partial skip is tested, so its documents are
dropped but its hits are not: in a constructed partial-response case run on
unmodified main, the merged {{numFound}} was 7 instead of 3 and
{{approximateTotalHits}} was 15 instead of 5. Second, when a shard sends no
{{approximateTotalHits}} header, the fallback adds the running per-query total,
which already includes the current shard, so two shards with 2 and 3 documents
merged to 7 instead of 5. Third, the {{hitCountIsExact}} fold is inverted: an
all-exact merge reports {{numFoundExact=false}} and an all-inexact merge
reports true, both confirmed by run on unmodified main.
The fix I recommend is to test the skip before accumulating, with the other two
corrections in the same change: the fallback should add the shard's own count
instead of the running total, and the fold should keep the merged flag true
only when every shard reports its count exact. Correcting the totals after the
skip fires instead would mean reconstructing what the skipped shard added, and
the exactness fold cannot be un-folded without per-shard tracking. One question
for the maintainer: when several sub-queries contribute, {{numFound}} is
currently the largest single sub-query count; should it instead count the
distinct documents that match any sub-query, and should {{numFoundExact}} be
false whenever more than one sub-query contributes?
AI agents assisted with research, implementation, review, and drafting. Nick
Shanin directed the work and takes responsibility for the above text.
> CombinedQueryComponent.mergeIds gives wrong numFound, approximateTotalHits
> and numFoundExact
> --------------------------------------------------------------------------------------------
>
> Key: SOLR-18541
> URL: https://issues.apache.org/jira/browse/SOLR-18541
> Project: Solr
> Issue Type: Bug
> Components: query
> Reporter: Nick Shanin
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> 🤖 _AI text below_ 🤖 _(reviewed by Nick Shanin)_
> CombinedQueryComponent.mergeIds gives wrong numFound, approximateTotalHits
> and numFoundExact when a shard is partial or errors.
> Three defects in the merged totals:
> 1. A partial shard's totals are added before its skip is tested. Its hits are
> counted but its documents are dropped. With a score-only sort, a partial
> shard with no per-query entry throws a NullPointerException.
> 2. The approximateTotalHits fallback, for a shard with no header, adds the
> running total instead of that shard's own count.
> 3. The numFoundExact fold was inverted. SOLR-18550 (PR #5172, merged
> 2026-10-23) fixed this on main, and it also set the rules for numFound and
> numFoundExact with several sub-queries.
> The fix is PR #5117, rebased on main. It tests the skip before any totals are
> added, uses the shard's own count in the fallback, and clears numFoundExact
> for skipped and errored shards.
> Still open: QueryComponent.mergeIds has the same leak as defect 1. A real
> timeAllowed exit can leave a shorter per-query list, and the unchecked access
> in CombinedQueryComponent is not changed by the PR.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
