bjchambers opened a new pull request, #23775:
URL: https://github.com/apache/datafusion/pull/23775
## Which issue does this PR close?
- Closes #23773.
## Rationale for this change
`array_any_value` reads the wrong value (or panics) when its input list
column
contains a non-null **empty** (length-0) element.
`general_array_any_value` guards null and all-null elements, but a non-null
empty element falls through to the no-nulls branch, which unconditionally
reads
`values[start]`:
- **interior empty list** → reads `values[start]`, i.e. the *next* element's
value (silently wrong data)
- **trailing empty list** (`start == values.len()`) → out-of-bounds slice →
panic `range end index N out of range for slice of length N-1`
The panic is easy to trigger in practice when the `array_any_value` output
flows
into a hash `RepartitionExec` (e.g. the value is used as an equi-join key):
repartitioning slices batches so an empty element can land at the end of a
values buffer, tripping the out-of-bounds read on a spawned task.
## What changes are included in this PR?
Guard the empty case explicitly in `general_array_any_value` — an empty list
has
no value to take, so the result is `NULL`.
Sibling functions in `extract.rs` were audited and are already safe:
`array_element` bounds-checks the index against `len`; `array_slice` /
`array_pop_front` / `array_pop_back` guard `len == 0`.
## Are these changes tested?
Yes:
- Kernel-level regression tests for `general_array_any_value`: an interior
empty
element (previously wrong value) and a trailing empty element (previously
panic).
- `array_any_value.slt` cases covering `List` and `LargeList` with interior
and
trailing empty elements.
## Are there any user-facing changes?
`array_any_value` now returns `NULL` for an empty list element instead of
returning the next element's value or panicking the query. No API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
