kazuyukitanimura opened a new pull request, #7149:
URL: https://github.com/apache/datafusion-comet/pull/7149
## Which issue does this PR close?
Closes #6532.
## Rationale for this change
#6532 reported that native `CASE WHEN` and `COALESCE` reconcile struct
fields by name, where Spark reconciles them by position and keeps the first
branch's field names. With case-insensitive resolution, case-distinct names
swapped by position could make the native common type pair an `INT` field with
a `DOUBLE` field and widen the `INT` to
`DOUBLE`. Native `to_json` then showed `{"x":7.0,"X":5.5}` instead of
`{"x":7,"X":5.5}`.
The behavior fix has already landed. #6827 changed `create_case_when` to
use the positional `merge_branch_types` helper that #6458 added for `IF`.
`COALESCE` is planned as `CASE WHEN x IS NOT NULL THEN x … ELSE last`, so it
goes through the same path. What remained from the issue was the regression
coverage it asks for: `CASE WHEN` was covered
for one branch order only, and `COALESCE` and nested array and map cases
were not covered.
## What changes are included in this PR?
Tests only, with no change to production code.
- `case_when.rs`: new unit test
`conditional_branches_reconcile_nested_case_variant_fields_positionally`.
- It reconciles case-distinct struct fields swapped by position, through
a bare struct, `array<struct>`, `map<string, struct>`, and three deeper
nestings (a struct holding a list, a list of maps, and a map holding a struct).
- It covers both branch orders, through `create_if_expr`,
`create_case_when`, and the `CASE WHEN x IS NOT NULL` shape that `COALESCE`
becomes.
- It asserts the result type (the first branch's names, merged
nullability) and the values, so `INT` is not widened to `DOUBLE`.
- `case_when_field_names.sql` and `coalesce_field_names.sql`: new SQL file
tests, one table each with batches where every row takes THEN, every row takes
ELSE, and rows are mixed.
- Both branch orders.
- Several `WHEN` branches / several `COALESCE` arguments.
- Arrays and maps, with the struct extracted by `[0]` / `['k']` because
native `to_json` does not accept arrays or maps.
- A nested struct and an array inside a map value.
## How are these changes tested?
- Rust: `cargo test -p datafusion-comet-spark-expr --lib
conditional_funcs` passes (30 of 30). With the positional struct merge in
`merge_branch_types` disabled, the new test and the existing positional test
both fail, so the test does catch the bug. `cargo fmt --check` is clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
