Xuebin Su has posted comments on this change. ( http://gerrit.cloudera.org:8080/22014 )
Change subject: IMPALA-13154: Update stats when loading an HDFS table ...................................................................... Patch Set 14: (3 comments) > Patch Set 12: > > (1 comment) Thanks for reviewing! http://gerrit.cloudera.org:8080/#/c/22014/1/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java File fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java: http://gerrit.cloudera.org:8080/#/c/22014/1/fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java@373 PS1, Line 373: if (part.getWriteId() >= 0) : thriftHdfsPart.setWrite_id(part.getWriteId()); : if (type == ThriftObjectType.FULL) { : thriftHdfsPart.setPartition_name(part.getPartitionName()); : thriftHdfsPart.setStats(new TTableStats(part.getNumRows())); : > > Note that when the table is a Hive ACID table or Iceberg V2 table, Got it. Thanks! http://gerrit.cloudera.org:8080/#/c/22014/12/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/22014/12/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@827 PS12, Line 827: // TODO(todd): would be good to log a summary of the loading process: > Just an idea: what about collecting the table/partition loading stats from Thanks! Added a FileMetadataStats in FileMetadataLoader. http://gerrit.cloudera.org:8080/#/c/22014/12/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java File fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java: http://gerrit.cloudera.org:8080/#/c/22014/12/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java@162 PS12, Line 162: } > Maybe it's more clear to set FileMetadataStats here and collect it inside F Thanks! Added a FileMetadataStats in FileMetadataLoader. -- To view, visit http://gerrit.cloudera.org:8080/22014 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e2eb503b0f61b1e6403058bc5dc78d721e7e940 Gerrit-Change-Number: 22014 Gerrit-PatchSet: 14 Gerrit-Owner: Xuebin Su <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Tue, 19 Nov 2024 02:38:08 +0000 Gerrit-HasComments: Yes
