Gabriel39 commented on code in PR #65262:
URL: https://github.com/apache/doris/pull/65262#discussion_r3575452848
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -596,11 +602,68 @@ public TableScan createTableScan() throws UserException {
this.pushdownIcebergPredicates.add(predicate.toString());
}
+ // Doris reads normal Iceberg table files in BE and applies column
pruning through scan range params.
+ // System tables are different: Iceberg SDK DataTask materializes rows
using the projected scan
+ // schema. Keep Doris required columns as the projection prefix so the
JNI reader can read rows by
+ // required ordinal, and append filter-only columns after them for
Iceberg residual evaluation.
+ if (isSystemTable) {
+ Schema projectedSchema =
getSystemTableProjectedSchema(expressions, scan.isCaseSensitive());
+ if (!projectedSchema.columns().isEmpty()) {
Review Comment:
**[P2] Keep projection pruning for zero-column scans**
When a system-table scan has no required slots (for example, `COUNT(*)`),
`projectedSchema` is empty and this branch skips `scan.project(...)`. The
Iceberg scan then keeps the full metadata schema. For `$files`/`$data_files`,
`ManifestReadTask.rows()` sees `readable_metrics` in that full projection and
constructs it for every row even though the JNI reader consumes zero fields. On
the `map<boolean, boolean>` tables covered by this PR, that can re-enter the
same `readable_metrics` materialization failure this change is intended to
avoid.
Please preserve pruning for this case as well—either use an empty SDK
projection if supported, or project a safe lightweight scalar placeholder while
keeping the JNI required-field count at zero. It would also be useful to add
`COUNT(*)` regression cases for both `$files` and `$data_files` on the
ORC/Parquet boolean-map tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
