[
https://issues.apache.org/jira/browse/LUCENE-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767564#comment-13767564
]
Robert Muir commented on LUCENE-4771:
-------------------------------------
By the way, I havent gone further here but I think this approach here can be
done even faster.
The patch here would simple reduce some hashing and ram-usage, but still be
slow for each query if there are many terms, because its 'aligning' terms
per-query.
instead, someone could simply make an OrdinalMap (it takes TermsEnum[]) of the
fields they are joining, which will "align" terms across all those
segments/fields into a global space, and the joining can just work on simple
ords.
this way you only align terms per-reopen instead of per-query. I think this
would be much faster.
> Query-time join collectors could maybe be more efficient
> --------------------------------------------------------
>
> Key: LUCENE-4771
> URL: https://issues.apache.org/jira/browse/LUCENE-4771
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/join
> Reporter: Robert Muir
> Attachments: LUCENE-4771_prototype.patch,
> LUCENE-4771-prototype.patch, LUCENE-4771_prototype_without_bug.patch
>
>
> I was looking @ these collectors on LUCENE-4765 and I noticed:
> * SingleValued collector (SV) pulls FieldCache.getTerms and adds the bytes to
> a bytesrefhash per-collect.
> * MultiValued collector (MV) pulls FieldCache.getDocTermsOrds, but doesnt
> use the ords, just looks up each value and adds the bytes per-collect.
> I think instead its worth investigating if SV should use getTermsIndex, and
> both collectors just collect-up their per-segment ords in something like a
> BitSet[maxOrd].
> When asked for the terms at the end in getCollectorTerms(), they could merge
> these into one BytesRefHash.
> Of course, if you are going to turn around and execute the query against the
> same searcher anyway (is this the typical case?), this could even be more
> efficient: No need to hash or instantiate all the terms in memory, we could
> do postpone the lookups to SeekingTermSetTermsEnum.accept()/nextSeekTerm() i
> think... somehow :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
