Gabriel39 opened a new pull request, #65934:
URL: https://github.com/apache/doris/pull/65934
### What problem does this PR solve?
Issue Number: N/A
Related PR: #65921
Problem Summary:
Projected PLAIN predicate columns could not use the raw fixed-value
predicate path. Doris materialized every selected predicate value and later
compacted the complete column even though the raw decoder had already
identified survivors.
This PR appends matching raw values to the projected column in the same
decoder pass. Predicate-only columns keep their existing placeholder behavior,
while residual predicates, delete predicates, unsupported physical/logical
types, and non-PLAIN encodings retain the existing materializing fallback.
The invariant is that the PLAIN decoder cannot rewind after raw predicate
evaluation, so a projected predicate column must receive its survivors before
the raw bytes are consumed.
Microbenchmark from #65921, Release build, warm cache, CPU 8, 10
repetitions, median values. The comparison used repeated A-B-B-A runs and reran
cases until both selected samples had CV <= 3%.
| Selectivity | CPU ns/raw row, master | CPU ns/raw row, PR | CPU
improvement | Real time, master | Real time, PR | Real improvement | CPU CV A/B
|
|---:|---:|---:|---:|---:|---:|---:|---:|
| 1% | 32.80 | 30.80 | 6.1% | 0.542 ms | 0.508 ms | 6.2% | 2.91% / 0.81% |
| 10% | 33.95 | 31.82 | 6.3% | 0.560 ms | 0.525 ms | 6.3% | 1.86% / 0.53% |
| 50% | 37.76 | 35.05 | 7.2% | 0.623 ms | 0.578 ms | 7.2% | 0.96% / 1.56% |
| 90% | 42.06 | 37.66 | 10.5% | 0.694 ms | 0.621 ms | 10.5% | 1.71% / 0.84% |
Benchmark command:
```shell
LD_LIBRARY_PATH=/mnt/disk6/common/jdk-17.0.16/lib/server \
taskset -c 8 be/output/lib/benchmark_test \
--benchmark_filter='^ParquetReader/predicate_scan/plain/null_10/alternating/sel_(1|10|50|90)/predicate_projected/width_32/predicate_0$'
\
--benchmark_min_time=1s \
--benchmark_repetitions=10 \
--benchmark_report_aggregates_only=true \
--benchmark_out=parquet-projected.json \
--benchmark_out_format=json
```
### Release note
None
### Check List (For Author)
- Test: Unit Test and Manual test
- `ParquetV2NativeDecoderTest.RawExprMapsNullableSparseRowsDirectly`
-
`ParquetScanTest.ProjectedPlainComparisonUsesPhysicalFilterAndProjectPath`
- Existing predicate-only, fallback, and residual-path Parquet scan tests
- All four `ParquetBenchmarkScenariosTest` cases
- Release warm-cache microbenchmark from #65921
- `build-support/clang-format.sh` and `build-support/check-format.sh`
- `build-support/run-clang-tidy.sh --build-dir be/ut_build_ASAN` was
attempted, but every changed translation unit was blocked by existing
master/toolchain errors: unmatched `NOLINTEND` in `be/src/core/types.h` and
missing `stddef.h` from the configured toolchain.
- Behavior changed: No. This only changes how eligible projected PLAIN
predicate survivors are materialized.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
