Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/22157 )
Change subject: IMPALA-11265: Part3: Group the Iceberg file descriptors by partition ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/22157/1/fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java File fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java: http://gerrit.cloudera.org:8080/#/c/22157/1/fe/src/main/java/org/apache/impala/catalog/IcebergContentFileStore.java@189 PS1, Line 189: Map<ByteBuffer, List<THdfsFileDesc>> thriftMap, Note to self: There seems to be an issue in local catalog mode, where even though the toThrift() populates the map properly where each bucket has a different partition hash, apparently on the receiver end the buckets have a very different (very long) hash binary keys that appear to be the same. As a result when populating the FileDescContainer all the FDs are put under the same bucket. Might be an issue how I pass the underlying byte[] to the throft object in toThrift. Not sure the ownership is handled correctly, might be reading memory junk on the receiver end. Will take a look. -- To view, visit http://gerrit.cloudera.org:8080/22157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I276d839335c0aa39fa31a06ce08588a91a313768 Gerrit-Change-Number: 22157 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Comment-Date: Wed, 04 Dec 2024 09:15:52 +0000 Gerrit-HasComments: Yes
