sunchao commented on code in PR #6826:
URL: https://github.com/apache/datafusion-comet/pull/6826#discussion_r4235682993
##########
spark/src/main/scala/org/apache/comet/expressions/CometCast.scala:
##########
@@ -119,10 +121,10 @@ object CometCast
inputs: Seq[Attribute],
binding: Boolean): Option[ExprOuterClass.Expr] = {
val cometEvalMode = evalMode(cast)
- cast.child match {
- case _: Literal =>
- exprToProtoInternal(Literal.create(cast.eval(), cast.dataType),
inputs, binding)
- case _ =>
+ foldLiteralCast(cast) match {
+ case Some(folded) =>
+ exprToProtoInternal(folded, inputs, binding)
+ case None =>
Review Comment:
[P2] Preserve ANSI rejection of decimal literals whose scale overflows. With
ANSI enabled and a Parquet table `t(id BIGINT)` containing `(0), (1)`, run
`SELECT CASE WHEN id = 1 THEN CAST('1e-2147483648' AS DECIMAL(10,2)) ELSE
CAST(0 AS DECIMAL(10,2)) END FROM t`. Spark raises `CAST_INVALID_INPUT` when
the branch is selected. This new failed-fold route instead reports the cast as
compatible and serializes it natively. In release/CI builds, the native
parser’s unchecked `i32` scale calculations wrap and return `0.00`, allowing an
invalid ANSI cast to produce a successful result. The parser flaw predates this
PR, but the base rejected this literal through `cast.eval()`, so its silent
acceptance is newly exposed here. Please use checked exponent/scale arithmetic
or keep affected failed folds on Spark, with conditional regression coverage.
Evidence: Local Spark 3.5.9, 4.1.3 and 4.2.0 Catalyst probes retained the
cast through ConstantFolding, returned `None` from the exact-head folding
helper, and produced `CAST_INVALID_INPUT` only for the selected branch. An
isolated Rust reproduction, with parser/helper bodies verified against this
checkout and compiled using `rustc -O`, returned `Ok(Some(0))` for both
`1e-2147483648` and `1.0e-2147483647` at precision 10, scale 2.
`native/Cargo.toml` disables overflow checks for release and CI. Source tracing
confirms `canCastFromString` admits DecimalType, scalar `spark_cast` reaches
`cast_string_to_decimal128_impl`, and `Ok(Some(0))` is accepted even in ANSI
mode. No end-to-end Comet execution was performed locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
