sunchao commented on PR #5846: URL: https://github.com/apache/datafusion-comet/pull/5846#issuecomment-5739186408
Addressed in 701e38a69 and rebased this PR onto current `main`. - `validate_list_lengths` now returns `SparkError::MapKeyValueDiffSizes`, so the existing shim produces Spark's structured runtime exception instead of a generic `CometNativeException`. - The unequal-length Spark regression now uses `checkSparkError(..., "_LEGACY_ERROR_TEMP_2128")`, which checks exception class, error class, SQLSTATE, and that no `CometNativeException` leaked through. - The ANSI null-key regression now puts both a divide-by-zero and a key/value length mismatch on the NULL-key row in the same batch, while the non-null rows remain valid. That pins key-first short-circuiting and null-before-length validation end to end. - Updated `map_funcs.md` to describe the nested guards and why one `AND` predicate is insufficient. For #5854: I checked its current head and it now preserves the nested guards. @peterxcli, my proposed landing order is this focused correctness fix first, then #5854 rebases and keeps its broader native map-builder implementation while preserving these guards and regressions. If #5854 lands first instead, this PR should be reduced to any remaining coverage/docs rather than retain a second wrapper. Local validation on the rebased head: - `cargo test -p datafusion-comet-spark-expr map_from_arrays` (6 passed) - `make core` - `./mvnw test -Dtest=none -Dsuites=org.apache.comet.CometMapExpressionSuite` (31 passed) - `cargo fmt --check`, `git diff --check`, and Maven Spotless/Scalastyle Fresh hosted CI is now queued for 701e38a69. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
