Houston Putman created SOLR-18017:
-------------------------------------
Summary: Encode all numeric docValues to be sortable
Key: SOLR-18017
URL: https://issues.apache.org/jira/browse/SOLR-18017
Project: Solr
Issue Type: Improvement
Reporter: Houston Putman
Currently when writing NumericDocValues (non-multivalued) we use
{{Float.floatToIntBits()}} or {{Double.doubleToLongBits()}} to encode the
float/double values for the NumericDocValues that require long values.
For multivalued fields, which use SortedNumericDocValues, we use
{{NumericUtils.floatToSortableInt()}} and
{{NumericUtils.doubleToSortableLong()}}. This gives us the added benefit of
being able to do range queries on these docValues because they are in sorted
order.
For the non-sortable single valued encoded longs, they will be in sorted order
except for when comparing positive and negative numbers. So when doing a range
query that doesn't cross 0, then there is really no issue. But range queries
for single valued floats and doubles cannot use the docValues range query
class, since all values are not in sorted order.
This is a pain, and the code to manage which encoding to use is a pain. The
Lucene utilities also expect that all floats and doubles are encoded in sorted
order which adds to the pain since we can't use those certain utilities for
single valued fields. It would be much easier to have all numeric docValues be
encoded in sorted order. There is a slight cost to doing the sorted order
encoding, but it is really just a bit operation.
The large cost here is that all numericDocValues are currently encoded not in
sorted order. So this would require re-indexing. We would need to introduce a
flag in Solr 10, and require users to reindex before upgrading to Solr 11 if
they are using single-valued point fields with docValues. (We would not be
doing this for TrieFields).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
