github-actions[bot] commented on code in PR #67962:
URL: https://github.com/apache/doris/pull/67962#discussion_r4045557735
##########
be/src/core/column/column_struct.cpp:
##########
@@ -218,24 +220,92 @@ void ColumnStruct::update_hash_with_value(size_t n,
SipHash& hash) const {
}
}
+namespace {
+
+// Invoke `func(run_start, run_end)` for every maximal run of rows in [start,
end) that are not
+// NULL according to `null_data`. An all-zero mask yields the single run
[start, end), so the
+// masked and unmasked range paths produce identical hashes and the same
number of calls.
+template <typename Func>
+void for_each_non_null_run(const uint8_t* __restrict null_data, size_t start,
size_t end,
+ Func&& func) {
+ size_t i = start;
+ while (i < end) {
+ while (i < end && null_data[i] != 0) {
+ ++i;
+ }
+ size_t run_start = i;
+ while (i < end && null_data[i] == 0) {
+ ++i;
+ }
+ if (run_start < i) {
+ func(run_start, i);
+ }
+ }
+}
+
+// Batch-hash every row through the field columns, then restore the rows that
are NULL at the
+// outer level. Surviving rows keep each field's batch hash contract (which
may differ from the
+// single/range contract, e.g. the width of a NULL default), and the field
columns are still
+// called once per column instead of once per row.
+template <typename HashType, typename Func>
+void batch_hash_skipping_null_rows(HashType* __restrict hashes, size_t rows,
+ const uint8_t* __restrict null_data, Func&&
hash_all_rows) {
+ DorisVector<HashType> saved(hashes, hashes + rows);
Review Comment:
[P2] Skip all-NULL batches before hashing hidden fields
When the outer map is all ones, `ColumnNullable` has already added the NULL
marker to every seed, but this helper still calls `hash_all_rows()` and only
then restores every entry. This is reachable when a mixed `IF(cond, NULL,
named_struct(...large ARRAY/MAP...))` retains branch values beneath its NULL
rows and a later filter or selection keeps only those rows. Both batch hooks
then walk all hidden elements even though no child result can survive, so
shuffle hashing becomes proportional to invisible payload. Please return before
allocating or dispatching to the children when the mask contains no zero, and
cover this with an all-NULL variable-length child using a child-call or
traversal oracle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
