Theodus opened a new issue, #24431: URL: https://github.com/apache/datafusion/issues/24431
### Describe the bug `ScalarValue::eq_array` is documented as an optimized equivalent of extracting an array element with `ScalarValue::try_from_array` and comparing the resulting scalars. However, for floating-point values `eq_array` uses IEEE equality while `ScalarValue::PartialEq` compares bit representations, producing inconsistent results: - Identical NaN bit patterns compare equal as `ScalarValue`s but unequal through `eq_array`. - `+0.0` and `-0.0` compare unequal as `ScalarValue`s but equal through `eq_array`. ### To Reproduce Compare these scalars with the corresponding Float64 array elements: ```text ScalarValue(NaN).eq_array([NaN], 0) = false ScalarValue(+0.0).eq_array([-0.0], 0) = true ``` In contrast, extracting those elements with `ScalarValue::try_from_array` and using `ScalarValue::eq` considers the identical NaN equal and the two signed zeros unequal. ### Expected behavior `ScalarValue::eq_array` should have the same floating-point equality semantics as `ScalarValue::PartialEq`: - Identical NaN bit patterns should compare equal. - Different NaN bit patterns should remain distinct. - `+0.0` and `-0.0` should remain distinct. ### Additional context `ScalarValue` implements `Eq` and `Hash`, and its float hashes and ordering are based on bit representations/total ordering. Changing `ScalarValue::PartialEq` to IEEE equality would violate `Eq` reflexivity for NaNs and require coordinated changes to hashing and ordering. The narrow fix is therefore to make the Float16, Float32, and Float64 branches of `eq_array` compare `to_bits()` values. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
