[
https://issues.apache.org/jira/browse/SOLR-16989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18066176#comment-18066176
]
Houston Putman commented on SOLR-16989:
---------------------------------------
[~magibney] did you actually record improvements in the case of
{{useDocValuesAsStored}}? In the PR you mentioned that there wasn't any
improvement for the ExportWriter. And according to SOLR-17539 this really
impacts Solr's memory footprint when using {{useDocValuesAsStored}}, so it
would be great to know the benefits so we can talk about if the tradeoffs are
worth it.
> Optimize and consolidate reuse of DocValues iterators for value retrieval
> -------------------------------------------------------------------------
>
> Key: SOLR-16989
> URL: https://issues.apache.org/jira/browse/SOLR-16989
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 10.0
> Reporter: Michael Gibney
> Assignee: Michael Gibney
> Priority: Major
> Fix For: 9.4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> DocValues are used for the purpose of arbitrary-order value retrieval mainly
> in two places:
> # {{ExportWriter}} (for the export handler)
> # {{SolrDocumentFetcher}} (for {{useDocValuesAsStored}})
> {{ExportWriter}} holds a simple optimization to reuse docValues iterators
> where possible (i.e., where docId doesn't decrease), but there is no such
> optimization for {{SolrDocumentFetcher}}: for every possible docValues return
> field, for every doc, a new docValues iterator is pulled (and GC'd). This is
> especially pathological in the case of many sparse fields, because the hit is
> incurred regardless of whether the return fields are actually present in any
> returned docs.
> This issue proposes unifying the docValues iterator caching for both these
> contexts, with the most immediate practical effect being improved latency of
> field retrieval for {{useDocValuesAsStored}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
