sunchao opened a new pull request, #6035: URL: https://github.com/apache/datafusion-comet/pull/6035
## Which issue does this PR close? No existing issue. This fixes the literal-cast planning failure described below. ## Rationale for this change Spark can retain a throwing ANSI cast of a literal inside a conditional branch that the query never visits. Comet evaluates literal casts while serializing the plan, so it can raise before Spark requests that branch. For example, with Parquet rows `(0, "0")` and `(1, "1")`, this query should return `0`: ```sql SELECT CAST(IF(id = 1, 'bad', value) AS INT) FROM cast_branch_rows LIMIT 1 ``` Spark pushes the cast into the conditional branches and retains the invalid literal cast for conditional evaluation. Eagerly evaluating it during Comet planning changes the successful query into an error. ## What changes are included in this PR? When evaluation of a literal cast throws a nonfatal exception, return a fallback with an explanation so Spark evaluates the expression when demanded. Successful literal casts keep the existing folding path. The regression checks the unvisited branch returns the first row while retaining a native Parquet scan. A second query demands the invalid branch and verifies both executions raise `CAST_INVALID_INPUT` with matching exception type and SQLSTATE. ## How are these changes tested? - Built the native library from current upstream sources with `make core`. - Full Spark 4.1 test compilation, Spotless, and Scalastyle pass. - The new regression fails on the unchanged upstream implementation with `CAST_INVALID_INPUT` during planning. - With the fix, the complete `CometScalarFunctionSuite` passes: 15 tests, no failures or aborted suites (`mvn test -Pspark-4.1 -Dtest=none -Dsuites=org.apache.comet.serde.CometScalarFunctionSuite`). - The `run-spark-4.1-tests` label requests the broader Spark SQL checks in CI. Codex assisted with adapting the existing fix and regression, source review, and validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
