wForget commented on code in PR #1295:
URL: https://github.com/apache/datafusion-comet/pull/1295#discussion_r1919477751
##########
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala:
##########
@@ -1928,6 +1928,23 @@ class CometExpressionSuite extends CometTestBase with
AdaptiveSparkPlanHelper {
}
}
}
+
+ test("hash functions with decimal input") {
+ withTable("t1", "t2") {
+ // when precision is less than or equal to 18, spark use decimal long
value to calculate hash.
+ // refer:
org.apache.spark.sql.catalyst.expressions.InterpretedHashFunction.hash
+ sql("create table t1(c1 decimal(18, 2)) using parquet")
+ sql("insert into t1 values(1.23), (-1.23), (0.0), (null)")
+ checkSparkAnswerAndOperator("select c1, hash(c1), xxhash64(c1) from t1
order by c1")
+
+ // TODO: comet hash function is not compatible with spark for decimal
with precision greater than 18.
+ // https://github.com/apache/datafusion-comet/issues/1294
+// sql("create table t2(c1 decimal(20, 2)) using parquet")
+// sql("insert into t2 values(1.23), (-1.23), (0.0), (null)")
+// checkSparkAnswerAndOperator("select c1, hash(c1), xxhash64(c1) from
t2 order by c1")
Review Comment:
Sorry, I have no idea to fix decimal to bytes consistency with spark yet, so
I just submitted this partial fix. I will change the PR description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
