[
https://issues.apache.org/jira/browse/SOLR-16289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696125#comment-17696125
]
Christine Poerschke commented on SOLR-16289:
--------------------------------------------
Picking up here from the
[https://github.com/apache/solr/pull/937#issuecomment-1448240364] comment, to
perhaps keep that pull request focused on the NullPointerException and
documentation and the discussion here focused on interleaving in SolrCloud.
It seems to me that there's (at least) two aspects:
* the _meaning_ of interleaving in a distributed setup i.e. how can and should
results be combined including what is correct (or good-enough)
* the _mechanics_ of how interleaving in a distributed setup could be done
As a starting point for thinking about the mechanics (and only the mechanics)
I'd like to refresh my understanding and outline how non-distributed
interleaving works:
* QueryComponent.prepare
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L384-L391]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L179-L197]
* QueryComponent.process
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L420-L427]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L327]
* QueryComponent calling SolrIndexSearcher
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L1637]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L710]
* SolrIndexSearcher to interleaving code
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L1759]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/search/AbstractReRankQuery.java#L60-L72]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/search/ReRankCollector.java#L118]
**
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/modules/ltr/src/java/org/apache/solr/ltr/interleaving/LTRInterleavingRescorer.java#L63]
The outline above is for non-distributed searches, {{prepare-then-process}} in
short.
For distributed searches we have
{{distributedProcess-handleResponses-finishStage}} in the search handler on the
"coordinator" node that receives the request plus multiple times
{{prepare-then-process}} in the search handlers on the nodes that receive
shard-requests from the coordinator.
*
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L477]
*
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L560]
*
[https://github.com/apache/solr/blob/releases/solr/9.1.1/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L566]
> [interleaving] transformer does not work in SolrCloud
> -----------------------------------------------------
>
> Key: SOLR-16289
> URL: https://issues.apache.org/jira/browse/SOLR-16289
> Project: Solr
> Issue Type: Improvement
> Components: contrib - LTR
> Affects Versions: 9.0
> Reporter: Naoto Minami
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> In SolrCloud, two-stage shard requests are processed. The first stage is to
> execute the query return unique keys and scores of documents. Then, in the
> second stage, collect fields’ values of merged top documents.
> LTRInterleavingTransformerFactory should be run in the first stage
> (ResponseBuilder.STAGE_EXECUTE_QUERY), because the
> LTRInterleavingScoringQuery knows which model is used in scoring. However,
> it’s run in second stage(ResponseBuilder.STAGE_GET_FIELDS) and
> LTRInterleavingRescorer#rescore is skipped in second stage.
> LTRInterleavingTransformerFactory cannot handle this case, so thrown
> NullPointerException when fl=[interleaving] is specified in SolrCloud. There
> is a same problem in LTRFeatureLoggerTransformerFactory. But, if interleaving
> is not used, LTRFeatureLoggerTransformerFactory falls back when feature
> vector cache is not hit (i.e. in second stage).
> I will fix the NullPointerException problem, but the underlying solution
> should be discussed. One of the solution of this problem is disable two-stage
> request by distrib.singlePass=true parameter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
