andygrove opened a new issue, #6522:
URL: https://github.com/apache/datafusion-comet/issues/6522
### Describe the bug
`signum(-0.0)` returns `0.0` in Comet, where Spark returns `-0.0`. Spark's
`Signum` calls `java.lang.Math.signum`, which returns its argument for either
zero. Comet maps `Signum` to DataFusion's `signum`
(`CometScalarFunction("signum")` in `QueryPlanSerde`), which returns `0.0` for
both zeros (`if v == 0.0 { 0.0 } else { v.signum() }` in `datafusion-functions`
55.1.0 `math/signum.rs`). The serde reports the expression as compatible, so
the query does not fall back.
Found by the float semantics sweep in #6521. Part of #6385.
### Steps to reproduce
```sql
CREATE TABLE t (d DOUBLE, f FLOAT) USING parquet;
INSERT INTO t VALUES (double('-0.0'), float('-0.0'));
SELECT signum(d), signum(f) FROM t;
```
| | Spark | Comet |
| --- | --- | --- |
| `signum(d)` | `-0.0` | `0.0` |
| `signum(f)` | `-0.0` | `0.0` |
Same on Spark 3.4, 3.5, 4.0, 4.1 and 4.2. `0.0`, NaN, the infinities and
ordinary values already match.
### Expected behavior
`signum(-0.0)` returns `-0.0`, as in Spark. The fix is to return the input
for a zero rather than `0.0`. That needs a Comet kernel or an upstream change
to DataFusion's `signum`. Once it lands, remove the `signum` entry from
`knownGaps` in `CometFloatSemanticsSuite`.
### Additional context
The difference is visible wherever the sign of zero is: `CAST(signum(d) AS
STRING)` gives `-0.0` against `0.0`, and `1 / signum(d)` gives `-Infinity`
against `Infinity`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
