viirya opened a new issue, #6825:
URL: https://github.com/apache/datafusion-comet/issues/6825
### Describe the bug
`CometCast.convert` folds every cast of a literal on the JVM while planning:
```scala
cast.child match {
case _: Literal =>
exprToProtoInternal(Literal.create(cast.eval(), cast.dataType), inputs,
binding)
```
Spark's `ConstantFolding` deliberately does not fold a foldable expression
that throws when it sits inside a conditional branch: it tags the node
`FAILED_TO_EVALUATE` and leaves it in the plan, because the branch may never
run (same behavior in 3.4, 3.5, 4.0 and 4.1). So under ANSI, a fallible cast of
a literal reaches Comet exactly in the case where it must not be evaluated
eagerly, and Comet's `cast.eval()` fails the whole query at planning time.
### Steps to reproduce
```sql
SET spark.sql.ansi.enabled=true;
CREATE TABLE t(id bigint) USING parquet;
INSERT INTO t VALUES (0), (1), (2);
SELECT id, CASE WHEN id = 5 THEN CAST('bad' AS BIGINT) ELSE id END FROM t;
```
Comet throws `[CAST_INVALID_INPUT] The value 'bad' of the type "STRING"
cannot be cast to "BIGINT"` from `CometCast$.convert`. The same happens for
e.g. `IF(id > 5, CAST(2147483648L AS INT), 0)` (`CAST_OVERFLOW`).
### Expected behavior
Spark returns `(0, 0), (1, 1), (2, 2)`, and raises the cast error only when
a row actually reaches the cast.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
