[
https://issues.apache.org/jira/browse/SOLR-17642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley resolved SOLR-17642.
---------------------------------
Resolution: Duplicate
I'm closing this as it's basically a duplicate/ramification of SOLR-17655. Not
just KNN queries but *all* queries. A simple/shorter term fix is happening in
SOLR-17673.
> Knn queries use multi threaded search unintentionally
> -------------------------------------------------------
>
> Key: SOLR-17642
> URL: https://issues.apache.org/jira/browse/SOLR-17642
> Project: Solr
> Issue Type: Bug
> Affects Versions: 9.7, 9.8
> Reporter: Varun Thacker
> Priority: Major
>
> First reported on
> [https://lists.apache.org/thread/xn1xg5szxhrs0sbcy4gmx4cvohy6flvh|https://lists.apache.org/thread/xn1xg5szxhrs0sbcy4gmx4cvohy6flvh,],
> Dr. Andreas Moll reported a failing query.
> I was able to reproduce is consistently locally on 9.7 and 9.8 using Dr.
> Andreas Moll's repro steps.
> There are two bugs in play here
> *The Solr side bug:*
> * Solr 9.7 added SOLR-13350 but disabled multi-threaded search by default. I
> can confirm the failing query follows this code path
>
> {code:java}
> if (!MultiThreadedSearcher.allowMT(pf.postFilter, cmd)) {
> log.trace("SINGLE THREADED search, skipping collector manager in
> getDocListNC");{code}
>
> * However when following that code path the query goes to
> AbstractKnnVectorQuery and this is where executor thread pool gets used
> {code:java}
> TimeLimitingKnnCollectorManager knnCollectorManager =
> new TimeLimitingKnnCollectorManager(
> getKnnCollectorManager(k, indexSearcher), indexSearcher.getTimeout());
> TaskExecutor taskExecutor = indexSearcher.getTaskExecutor();
> {code}
> * This executor was added in SOLR-13350 in CoreContainer(
> indexSearcherExecutor) . If you follow the SolrIndexSearcher code that calls
> {code:java}
> In SolrIndexSearcher.java
> super(wrapReader(core, r),
> core.getCoreContainer().getIndexSearcherExecutor());{code}
> {code:java}
> In IndexSearcher.java
> this.executor = executor;
> this.taskExecutor =
> executor == null ? new TaskExecutor(Runnable::run) : new
> TaskExecutor(executor);{code}
> I think the idea in SOLR-13350 was create an indexSearcherExector, but only
> use it when multiThreaded=true as a query param (default=false). The fact we
> initialised indexSearcherExector means all KNN queries use that thread pool
> and there is no way to disable that.
>
> *Now the Lucene Bug:*
> [~benwtrent] noted on
> [https://lists.apache.org/thread/ftm6f32f2nsoys6noxx8p2ygpwnlfjc9] that
> Multi-threaded vector search over multiple segments can lead to inconsistent
> results
>
> So the fact Solr by default always uses multi threaded search 9.7+ triggers
> the lucene bug. But I think in Solr we should also have a way to not use
> multi-threaded KNN like we do for other queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
