sunchao commented on code in PR #4987:
URL: https://github.com/apache/datafusion-comet/pull/4987#discussion_r4433048096
##########
spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala:
##########
@@ -2073,4 +2073,446 @@ class CometWindowExecSuite extends CometTestBase {
}
}
}
+
+ //
---------------------------------------------------------------------------
+ // Tests for issue #4834: RANGE frame with explicit PRECEDING/FOLLOWING
offset
+ // on DECIMAL ORDER BY. Comet previously fell back to Spark because
DataFusion
+ // rejected boundary arithmetic that widened precision (e.g. Decimal(10,0) +
+ // offset -> Decimal(11,0) vs current Decimal(10,0), "Uncomparable values").
+ // The upstream fix (apache/datafusion#22174, closes #22113) makes
+ // ScalarValue::partial_cmp ignore precision for Decimal128 when scales
match,
+ // and was first released in DataFusion 54.0.0. These tests pin the behavior
+ // with `checkSparkAnswerAndOperator` so results must match Spark AND run
+ // natively.
+ //
---------------------------------------------------------------------------
+
+ private def writeDecimalWindowFixture(dir: java.io.File, decimalType:
String): Unit = {
+ (0 until 30)
+ .map { i =>
+ // Fractional part rotates 0.00/0.25/0.50/0.75 so RANGE peers vary; c
+ // stays as the aggregate payload.
+ val whole = i % 10
+ val frac = (i % 4) * 25
+ (i % 3, f"$whole%d.$frac%02d", i * 2)
+ }
+ .toDF("a", "d_str", "c")
+ .selectExpr("a", s"CAST(d_str AS $decimalType) AS d", "c")
+ .repartition(3)
+ .write
+ .mode("overwrite")
+ .parquet(dir.toString)
+ }
+
+ test("window: RANGE BETWEEN N PRECEDING AND N FOLLOWING with DECIMAL ORDER
BY") {
Review Comment:
Several tests rewrite the same Parquet input and vary only the SQL frame.
Could the ordinary preceding/following cases share one table in
`windows/window_functions.sql`? The runner compares rows and checks native
execution. The tests that specifically distinguish `CometWindowExec` from a
fallback at the precision boundary still have a reason to stay in Scala.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
