Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/21019 )
Change subject: IMPALA-12487: Skip reloading file metadata for ALTER_TABLE events with trivial changes in StorageDescriptor ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/21019/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21019/1//COMMIT_MSG@8 PS1, Line 8: trivial changes in StorageDescriptor high level design question (which I should have asked in the Jira earlier): what changes do we expect in StorageDescriptor do happen frequently? If these changes are expected to be very rare, then I am not sure that it worth the added complexity and risk to optimize them. IMPALA-11534 was critical as some table properties can change on every insert, which can be very frequent. Is there a reason why StorageDescriptor should change regularly, e.g. every day? http://gerrit.cloudera.org:8080/#/c/21019/1//COMMIT_MSG@15 PS1, Line 15: skipped for all other changes in S It may be safer to have a set of members that are safe to ignore. What happens if a member is added to StorageDescriptor in Hive? It sounds unlikely to add a new member that changes the expected directory structure, but I also can't exclude it completely. It seems better to reload files for unknown members, and later add them to the ignore list if it turns out that they don't need reload. -- To view, visit http://gerrit.cloudera.org:8080/21019 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6fd9a9504bf93d2529dc7accbf436ad83e51d8ac Gerrit-Change-Number: 21019 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Thu, 08 Feb 2024 10:37:11 +0000 Gerrit-HasComments: Yes
