Gabriel39 opened a new pull request, #65495:
URL: https://github.com/apache/doris/pull/65495
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
Parquet COUNT pushdown uses one representative leaf to count the top-level
null state of a complex column. When a STRUCT contains an ARRAY or MAP
descendant, the representative levels are repeated. The previous code compared
repetition levels with the STRUCT root repetition level, which is zero, skipped
every level entry, and triggered a `DORIS_CHECK` that aborted the BE. It also
derived the non-null threshold from the repeated leaf definition level and
could count a NULL struct as non-null.
This change identifies top-level rows with repetition level zero and uses
the root schema's nullable definition level to determine whether the top-level
complex value is non-null. Empty and NULL collections inside a non-null STRUCT
remain valid STRUCT rows.
### Release note
Fix BE aborts and incorrect COUNT results for Parquet STRUCT columns
containing ARRAY or MAP descendants.
### Check List (For Author)
- Test: Unit Test
-
`NewParquetReaderTest.CountStructWithRepeatedChildUsesTopLevelRowBoundaries`
- Existing STRUCT, LIST, and MAP COUNT pushdown tests
- Behavior changed: Yes. COUNT pushdown uses top-level row/null semantics
for repeated descendants.
- Does this need documentation: No
Validation on the designated Linux build host:
- `build-support/check-format.sh`
- Targeted BE unit tests: 4 tests passed
- `run-clang-tidy.sh`: test file has no warnings; production analysis
reports pre-existing full-file diagnostics and `jni-util.h` toolchain static
assertions unrelated to this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
