Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22871 )
Change subject: IMPALA-12162: Checksum files before lock in INSERT ...................................................................... Patch Set 8: (3 comments) http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7348 PS7, Line 7348: Map<String, FileMetadata> fileMetadata = getFileMetadata( : feFsTable, update.getUpdated_partitions().values(), catalogTimeline); > That potentially adds more overhead to creating and tearing down the thread Done http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7857 PS7, Line 7857: String checksum; > Now that we have FileMetadata as container class, can we keep checksum as b Yes. Saving the FileChecksum, which optimizes its own storage. http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7910 PS7, Line 7910: if (metadata == null) continue; > This isn't a good idea because then FilesAdded and FilesAddedChecksum might Done -- To view, visit http://gerrit.cloudera.org:8080/22871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I18f9686f5d53cf1e7c384684c25427fb5353e2af Gerrit-Change-Number: 22871 Gerrit-PatchSet: 8 Gerrit-Owner: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Comment-Date: Wed, 21 May 2025 00:28:08 +0000 Gerrit-HasComments: Yes
