Mryange opened a new pull request, #66765:
URL: https://github.com/apache/doris/pull/66765
[opt](exec) optimize nullable predicate scan performance
### What problem does this PR solve?
Issue Number: N/A
Related PR: #66688
Problem Summary:
Nullable predicate scans repeatedly traverse all-zero null maps even when a
decoded batch contains no NULL values. In addition, `Roaring::cardinality()` is
recomputed for every scan batch although the row bitmap is unchanged after
iterator initialization.
Root cause: the segment iterator does not preserve the decoded batch's null
state, and it queries row bitmap cardinality in the per-batch path.
This PR:
- vectorizes nullable comparisons by replacing short-circuit `&&` with
bitwise `&`;
- propagates `batch_has_null` from column decoding and evaluates null-free
comparison batches on the nested column;
- filters nested values directly for null-free output and creates an
all-zero output null map;
- caches the final row bitmap cardinality during lazy initialization.
Unknown null states, batches containing NULL values, and unsupported
predicates retain the original nullable path.
Local manual tests used a 2-billion-row, single-tablet table with one
scanner and profile/cache disabled. The nullable scans averaged 4.648s versus
6.417s in StarRocks for `>=`, 3.907s versus 4.127s for `BETWEEN`, and 2.314s
versus 2.660s for equality. A four-scan `UNION ALL` query averaged 4.654s in
Doris versus 5.587s in StarRocks.
### Release note
None
### Check List (For Author)
- Test
- [x] Manual test (details included above)
- [ ] Regression test
- [ ] Unit Test
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
