linrrzqqq opened a new pull request, #70777:
URL: https://github.com/apache/doris/pull/70777
### Background
A `ColumnNullable` stores its values and its NULL flags in separate columns.
An expression such as `NULLIF(k, 1)` can share the values column with `k`. For
a row where `k = 1`, the expression reads as NULL while the query can still
return the original `1`.
### What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Bugfix
Consider a join on `NULLIF(l.k, 1) <=> NULLIF(r.k, 2)` that also selects
`l.k, r.k`. The row with keys `1` and `2` should match and return `(1, 2)`.
Preparing the probe key could overwrite the shared `1` with `0`, changing the
projected value. Load conversion had the same ownership problem one level up:
deduplicating a Map in the output could remove entries from the input Block
because both Blocks referenced the same Map object.
### Details
- **Frequency:** The write is visible when another key, output slot or Block
still references the same column.
- **Root cause:** Key preparation wrote through borrowed pointers. Filtering
checked only the nested column's reference count, which can be one even when
two Blocks share its Nullable parent. Load validation also replaced child
pointers inside shared parent columns.
- **Why existing tests missed it:** A filter can select the correct rows
even after overwriting the source bytes under NULL. A load can produce the
correct deduplicated Map even after modifying the input Map. Checking only the
result misses both writes.
- **Fix:**
- For a single non-string probe key whose NULLs must match, obtain a
writable key with `IColumn::mutate`, set its NULL positions to the default
value, and keep it alive until the join finishes probing that input Block.
- When packing fixed-width keys, write zero bytes for NULL directly into
the packed key. If two fields share the same input, packing `(NULL, 7)` now
preserves the second field's `7`.
- Before combining a nullable filter's value bytes with its NULL flags,
check `ColumnNullable::is_exclusive()`. If the column is shared, copy the value
bytes and apply the mask to the copy.
- Pass the destination `ColumnPtr` by reference during load validation so
a converted child can be installed in the correct output field. Deduplicate Map
keys on a cloned outer Map, rebuild an Array or copy a Struct when a child
changes, and obtain writable decimal data before writing a rounded value.
- **Other paths and branches:** The fixed-width key change also applies to
grouping on multiple columns that share input data.
### Testing
The BE and FE build and 67 ASAN unit tests passed. The tests check that
packing keys preserves the source values, filtering preserves the source byte
under NULL, and deduplicating a Map with entries `(7, 10), (7, 20)` keeps `(7,
20)` in the output and both entries in the input.
Both P0 suites passed locally. `test_hash_join_probe_side_zero_copy` checks
the projected `(1, 2)` row in the join above. `test_string_len_complex` checks
string length handling in nested columns during load.
### Release note
Null-safe joins preserve the original values of projected join columns, and
converting nested data for a load preserves values shared with other output
fields.
### Check List (For Author)
- Behavior changed: Yes (see Details).
- Does this need documentation: No.
- Pre-submission status: Build and tests passed. Clang-tidy reports function
size and cognitive complexity in
`OlapTableBlockConvertor::_internal_validate_column`.
### AI Disclosure
- Tool: OpenAI Codex, GPT-6.
- Generated: Implementation, tests and commit description.
- Human review: Pending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]