sunchao commented on code in PR #5393:
URL: https://github.com/apache/datafusion-comet/pull/5393#discussion_r3812048072
##########
spark/src/test/resources/sql-tests/expressions/array/shuffle.sql:
##########
@@ -124,7 +124,7 @@ statement
INSERT INTO test_shuffle_double VALUES
(array(1.1, 2.2, 3.3, 4.4, 5.5)),
(NULL),
- (array(CAST('NaN' AS DOUBLE), CAST('Infinity' AS DOUBLE), CAST('-Infinity'
AS DOUBLE), 0.0, -0.0))
+ (array(CAST('NaN' AS DOUBLE), CAST('Infinity' AS DOUBLE), CAST('-Infinity'
AS DOUBLE), 0.0, double('-0.0')))
Review Comment:
[P2] Keep the unseeded permutation check deterministic
Could this mixed signed-zero case use a comparison that does not depend on
the order of equal zeros? The query below compares `sort_array(shuffle(arr))`
exactly, but Spark's nullable-array comparator treats `0.0` and `-0.0` as
equal. Its stable sort therefore preserves their randomly shuffled order. The
Spark and Comet runs resolve independent seeds, and the result comparator
distinguishes the zero bits. With the new row, repeated Spark 4.1.1 executions
produce both orders, while the baseline's two positive zeros were stable. This
introduces a flaky SQL test. Sorting the string representations, for example
`sort_array(transform(shuffle(arr), x -> cast(x AS string)))`, preserved both
zero signs and produced one result across seeds 0 through 11 in a Spark-only
probe. Could we use that kind of canonical multiset comparison here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
