Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/22559 )
Change subject: IMPALA-11402: Add limit on files fetched by a single getPartialCatalogObject request ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/22559/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22559/1//COMMIT_MSG@16 PS1, Line 16: partition level when it's too big. In this case, some partitions will be Reword this to so not say partitions are missing since that implies some bad state. Better to say that the next subset of partitions is returned on each called. http://gerrit.cloudera.org:8080/#/c/22559/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/22559/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@1105 PS1, Line 1105: List<Long> remainingIds = Lists.newArrayListWithCapacity( > These are partition ids. Before sending the request, coordinator don't know Thanks for the clarification. Since the partial fetch is delimited by partition boundary, what will happen of there are more than catalog_partial_fetch_max_files in a single partition? It should [attempt to] return one partition in that case to avoid errors. -- To view, visit http://gerrit.cloudera.org:8080/22559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb13fec20de5a17e7fc33613ca5cdebb9ac1a1e5 Gerrit-Change-Number: 22559 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Comment-Date: Wed, 19 Mar 2025 14:52:04 +0000 Gerrit-HasComments: Yes
