neilconway opened a new issue, #25293:
URL: https://github.com/apache/datafusion/issues/25293
### Describe the bug
Unnesting a struct column treats the child arrays as independent of the
struct's own validity, in both the planner and the executor:
- `get_unnested_columns` copies each child's declared nullability into the
output schema without checking whether the parent struct is nullable.
- `flatten_struct_cols` returns `struct_arr.columns()` verbatim, without
applying the struct's null buffer.
### To Reproduce
This can result in unnesting values from a "null" struct:
```sql
-- One row holding a fully valid struct.
create table base as select named_struct('a', 1, 'b', 'x') as s;
-- nullif on a column goes through arrow's nullif kernel, which only flips
the
-- struct's validity bit. The child arrays still hold 1 and 'x'.
create table t2 as select nullif(s, s) as s from base;
select s from t2;
-- NULL
select s is null from t2;
-- true
select unnest(s) from t2;
-- 1 x
```
```sql
select unnest(arrow_cast(NULL, 'Struct("a": non-null Int32)'));
```
yields:
```
Arrow error: Invalid argument error: Column
'__unnest_placeholder(arrow_cast(NULL,Utf8("Struct("a": non-null Int32)"))).a'
is declared as non-nullable but contains null values
```
### Expected behavior
- Unnesting a NULL struct row yields NULL for every output column.
- Every output column of a struct unnest is nullable whenever the parent
struct is nullable, matching the rule get_field already follows for nested
access.
i.e., first query above should return `NULL NULL`, second query should
return `NULL` instead of an error.
### Additional context
Similar to #25120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
