viirya opened a new issue, #6759:
URL: https://github.com/apache/datafusion-comet/issues/6759
### Describe the bug
`least` over structs or arrays orders a null inside the value after every
other value. Spark orders it before every other value, in `least` as in
`greatest`, so Comet can return a different value from Spark.
Comet uses its own `SparkGreatestLeast` only for floats and nested types
with a float leaf. Other nested types go to DataFusion's `least`, whose
comparator uses `nulls_first: false`.
### Steps to reproduce
```sql
SELECT id, least(named_struct('a', CAST(id AS INT), 'b', 1),
named_struct('a', IF(id = 3, NULL, 0), 'b', 3))
FROM range(4);
```
With `spark.comet.exec.range.enabled=true`,
`spark.comet.sparkToColumnar.enabled=true`, and
`spark.comet.sparkToColumnar.supportedOperatorList=Range`, Comet runs this
natively. For `id = 3` Spark returns `[null, 3]` and Comet returns `[3, 1]`.
`least(array(CAST(id AS INT), 1), array(IF(id = 3, NULL, 0), 3))` behaves the
same way.
### Expected behavior
A null inside a struct or array ranks before every other value, so `least`
returns `[null, 3]` for `id = 3`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
