aoto-tech opened a new issue, #25265:
URL: https://github.com/apache/datafusion/issues/25265
### Describe the bug
I found that `array_distance`, `cosine_distance`, and `array_normalize`
directly sum squared `Float64` values. The intermediate squares can overflow or
underflow even when the correct result is representable.
This can turn nonzero distances into zero, identical vectors into `NaN`, and
valid normalized vectors into zero vectors.
### To Reproduce
Tested on 55.1.0 and `main` at `9082d6b10`.
```sql
SELECT array_distance([CAST(1e-200 AS DOUBLE)], [CAST(0 AS DOUBLE)]);
-- actual: 0.0; expected: 1e-200
SELECT cosine_distance(
[CAST(3e200 AS DOUBLE), CAST(4e200 AS DOUBLE)],
[CAST(3e200 AS DOUBLE), CAST(4e200 AS DOUBLE)]
);
-- actual: NaN; expected: 0.0
SELECT array_normalize([CAST(3e200 AS DOUBLE), CAST(4e200 AS DOUBLE)]);
-- actual: [0.0, 0.0]; expected: [0.6, 0.8]
```
### Expected behavior
Finite inputs should not produce zero, `NaN`, or a zero vector when the
mathematical result is finite and representable.
### Additional context
The same unscaled sum-of-squares pattern appears in
[`array_distance`](https://github.com/apache/datafusion/blob/9082d6b10c29b72d56bede3d8e353d9d61fde542/datafusion/functions-nested/src/distance.rs#L192-L201),
[`cosine_distance`](https://github.com/apache/datafusion/blob/9082d6b10c29b72d56bede3d8e353d9d61fde542/datafusion/functions-nested/src/cosine_distance.rs#L200-L214),
and
[`array_normalize`](https://github.com/apache/datafusion/blob/9082d6b10c29b72d56bede3d8e353d9d61fde542/datafusion/functions-nested/src/array_normalize.rs#L179-L195).
I couldn't find an existing issue for this behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
