[
https://issues.apache.org/jira/browse/SOLR-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-9285:
---------------------------
Attachment: SOLR-9285.patch
The root cause of the AIOOBE is that when a doc is found in the {{UpdateLog}},
the RTG component explicitly passes -1 to {{DocTransformer.transform()}}, and
currently {{ValueSourceAugmenter.transform()}} doesn't account for this
possibility when using it's IndexReader to lookup the FunctionValues for the
document.
Obviously, even if it did do some sanity checking for negative docids, that
wouldn't really address the root of the problem -- getting usable
FunctionValues.
The RTG Component already has a code path where it sometimes uses
{{ulog.openRealtimeSearcher()}} when doc are found in the update log, instead
of returning the document directly from the ulog command -- but that code path
is currently confined to situations where {{fq}} params are included in the RTG
request (so it can evaluate them against the Realtime Searcher to ensure the
doc should really be returned)
----
I'm attaching a straw man patch that:
* refactors the {{ulog.openRealtimeSearcher()}} logic in RTG Component to also
apply when there are transformers
* Adds a new light weight (private) {{RTGResultContext}} class for wrapping
realtime {{SolrIndexSearcher}}s for use with transformers.
* Fixes {{ValueSourceAugmenter.setContext}} to use
{{ResultContext.getSearcher()}} instead of the one that comes from the request
** Independent of anything else, this seems like a bug (it certainly violates
the javadocs of {{ResultContext.getRequest()}}, even if it was getting the same
SolrQueryRequest via a more round about way)
This patch fixes all existing {{@AwaitsFix(SOLR-9285)}} tests, and doesn't seem
to cause any new test failures in the code base, but i don't think we should
commit it as is for the simple reason that it seems like it is overkill: Simple
transformers like RenameFieldTransformer can operate just fine on docs from the
UpdateLog w/o any need for a (realtime) SolrIndexSearcher.
As noted in the patch with a {{nocommit}} I think what would be best is to
enahnce this patch by adding a {{boolean needsSolrIndexSearcher()}} method to
the {{DocTransformer}} API, which defaults to {{false}} but any transformers
like {{ValueSourceAugmenter}} that actually _need_ access to a
{{SolrIndexSearcher}} in their {{setContext}} method could ask for it, and the
code in this patch that sets {{mustUseRealtimeSearcher = true}} anytime there
is a transformer could be modified to instead use
{{transformer.needsSolrIndexSearcher()}}
Questions/comments/concerns?
----
Reviewing the RTG Component code also makes me realize that in general we
should have more RTG+transformer tests which:
* use {{fq}} params
* ask for uncommited ids both immediately after adding them (before a realtime
searcher may have been opened) *AND* "after" a realtime searcher has been
opened (for whatever reason)
* ask for multiple ids in a single request (to ensure nothing broken in the
looping logic)
** mixing commited, uncommited, and uncommited but in an already re-opened
realtime searcher
** mixing them in various orders, since that affects *when* a realtime searcher
might be opened
> ArrayIndexOutOfBoundsException when ValueSourceAugmenter used with RTG on
> uncommitted doc
> -----------------------------------------------------------------------------------------
>
> Key: SOLR-9285
> URL: https://issues.apache.org/jira/browse/SOLR-9285
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Hoss Man
> Attachments: SOLR-9285.patch
>
>
> Found in SOLR-9180 testing.
> Even in single node solr envs, doing an RTG for an uncommitted doc that uses
> ValueSourceAugmenter (ie: simple field aliasing, or functions in fl) causes
> an ArrayIndexOutOfBoundsException
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
