[
https://issues.apache.org/jira/browse/SOLR-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17726008#comment-17726008
]
Chris M. Hostetter commented on SOLR-16693:
-------------------------------------------
{quote}We might end up adding a check before each faceting step (e.g. between
fields) instead of doing a slow term-by-term check.
{quote}
That's not really viable for situations where the "slow" code is a tight/deep
loop inside the lucene code (like with spellchecker, or complex query
rewriting).
The entire reason ExitableDirectoryReader was created was precisely to help
ensure that *any* usage of the the underlying index (search, facet, spell,
query rewrite, MLT, etc...) would "fail fast" if the {{timeAllowed}} was
exceeded – w/o it {{timeAllowed}} is virtually useless for most code paths (as
it was 10 years ago when it really only triggered during document collection).
"Are we there yet?" type checks in the solr level will *never* be as good as
ExitableDirectoryReader.
* how bad exactly is the (new) performance penalty?
* do you have any counterpoints to my strawman proposal (above) to try and get
{{timeAllowed}} to the point it's useful again?
> Use TimeLimitingBulkScorer; stop using ExitableDirectoryReader
> --------------------------------------------------------------
>
> Key: SOLR-16693
> URL: https://issues.apache.org/jira/browse/SOLR-16693
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 9.2
> Reporter: David Smiley
> Assignee: David Smiley
> Priority: Blocker
> Fix For: 9.3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> While reviewing Lucene 9.5 changes coming into Solr, I noticed some changes
> relating to the ability to specify timeAllowed in Solr on a search query.
> Solr uses both ExitableDirectoryReader and TimeLimitingCollector from Lucene
> for this (complementary to each other). Unfortunately, changes in Lucene
> will make the cost of ExitableDirectoryReader wrapping happen for all queries
> into Solr, even those not using timeAllowed. Options to keep EDR aren't good
> -- fork it basically. Anecdotally, I think I've heard the overhead is not
> trivial and my intuition thinks likewise. Meanwhile, Lucene 9.3 added a new
> TimeLimitingBulkScorer which even gets first class integration into
> IndexSearcher which has a timeout. It's been incrementally improved, and I
> really like its approach, probable performance, and simplicity. It should be
> straightforward to integrate this into SolrIndexSearcher and also only do so
> for queries specifying timeAllowed. I'm not sure TimeLimitingCollector
> offers much value to using TLBS other than additional precision on
> timeAllowed at some cost to unselective queries.
> I think doing this should block Solr 9.2 using Lucene 9.5. Alternatively,
> someone might benchmark the state of things and see that things aren't so bad
> as they may seem. But that takes work too.
> [1] QueryTimeout.isTimeoutEnabled is gone:
> https://github.com/apache/lucene/pull/11954
> [2] TimeLimitingBulkScorer in LUCENE-10151
> https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/TimeLimitingBulkScorer.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
