Hello Fang-Yu Rao, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21178
to look at the new patch set (#2).
Change subject: IMPALA-12929: Skip loading HDFS permissions in local-catalog
mode
......................................................................
IMPALA-12929: Skip loading HDFS permissions in local-catalog mode
HDFS file/dir permissions are not used at all in local catalog mode - in
LocalFsTable, hasWriteAccessToBaseDir() always returns true and
getFirstLocationWithoutWriteAccess() always returns null.
However, in catalogd, we still load them (in single thread for a table!)
which could dominant the table loading time when there are lots of
partitions. Note that the table loading process in catalogd is the same
no matter what catalog mode is in used. The difference between catalog
modes is mainly in how coordinators get metadata from catalogd. Local
catalog mode is turned on by setting --catalog_topic_mode=minimal on
catalogd and --use_local_catalog=true on coordinators.
This patch skips loading HDFS permissions on catalogd when running in
local catalog mode. We can revisit it in IMPALA-7539.
Tests:
- Ran CORE tests
Change-Id: I5baa9f6ab0d3888a78ff161ae5caa19e85bc983a
---
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
2 files changed, 8 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/21178/2
--
To view, visit http://gerrit.cloudera.org:8080/21178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5baa9f6ab0d3888a78ff161ae5caa19e85bc983a
Gerrit-Change-Number: 21178
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
