Gabriel39 opened a new pull request, #65498:
URL: https://github.com/apache/doris/pull/65498
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: FileScannerV2 cloned its table-level conjuncts only when
TableReader was
initialized. Runtime filters arriving after scanner open were visible to the
scanner but never
reached subsequent split readers. Query results remained correct because
scanner-level evaluation
was still applied, but Parquet and ORC metadata pruning, native file
filtering, and JNI block
filtering could not benefit from late runtime filters.
This change passes a freshly cloned conjunct snapshot into every split
regardless of the partition
pruning setting. TableReader replaces its initial snapshot during split
preparation, and JNI
readers prepare and open the fresh snapshot per split. Hybrid readers
forward the same split
options to their active native or JNI child reader. V1 behavior is unchanged.
### Release note
Enable late runtime filters to participate in FileScannerV2 file-level
filtering for subsequent
splits.
### Check List (For Author)
- Test: Unit Test
- `TableReaderTest.PrepareSplitReplacesInitialConjunctSnapshot`
- `FileScannerV2Test.RewriteSlotRefsToGlobalIndexMatrix`
- Behavior changed: Yes. Subsequent FileScannerV2 splits use the latest
runtime-filter snapshot for
native and JNI filtering.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
