rkrishn7 opened a new pull request, #27501: URL: https://github.com/apache/datafusion/pull/27501
## Which issue does this PR close? - Related to #27498. Found during review of #27499. ## Rationale for this change Two `GroupValues::intern` paths can panic: - `GroupValuesColumn::emit(EmitTo::All)` replaces the column builders but keeps the hash map and collision lists. Interning a key seen before the emit finds a stale entry and indexes the empty builders: `index out of bounds: the len is 1 but the index is 2`. In-tree callers call `clear_shrink` after `emit(EmitTo::All)` or stop, so only code that reuses a `GroupValues` after a full emit hits this. - `ArrowBytesViewMap::insert_if_new` compares a long input with a stored entry that has the same hash and 4-byte prefix without checking the length. If the stored entry is inline, its bytes are read as a buffer index and offset: `range start index 1499027801 out of range for slice of length 0`. The values can't compare equal, so this never returns a wrong group, but it needs a 64-bit hash collision to happen and then panics. ## What changes are included in this PR? - `GroupValuesColumn::emit(EmitTo::All)` clears the hash map and collision lists along with the column builders. - `ArrowBytesViewMap::insert_if_new` skips entries whose length differs from the input, as `ArrowBytesMap` already does. ## What is the testing strategy for this PR? Two unit tests that fail on `main` and pass with this PR: - `intern_after_emit_all_starts_new_groups` interns, emits everything, interns again, and checks the new group ids in both streaming and vectorized modes. - `test_long_value_colliding_with_inline_value` interns an inline value and then a long value with the same prefix. Hashes only collide with the `force_hash_collisions` feature, which the extended CI job enables. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
