Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22789
Change subject: IMPALA-13631: alterTableOrViewRename shouldn't hold catalog versionLock during external RPCs ...................................................................... IMPALA-13631: alterTableOrViewRename shouldn't hold catalog versionLock during external RPCs Catalog versionLock is a lock used to synchronize reads/writes of catalogVersion. It can be used to perform atomic bulk catalog operations since catalogVersion cannot change externally while the lock is being held. All other catalog operations will be blocked if the current thread holds the lock. So it shouldn't be held for a long time, especially when the current thread is invoking external RPCs for a table. CatalogOpExecutor.alterTable() is one place that could hold the lock for a long time. If the ALTER operation is a RENAME, it holds the lock until alterTableOrViewRename() finishes. HMS RPCs are invoked in this method to perform the operation, which might take an unpredictive time. The motivation of holding this lock is that RENAME is implemented as an DROP + ADD in the catalog cache. So this operation can be atomic. However, that doesn't mean we need the lock before operating the cache in CatalogServiceCatalog.renameTable(). We actually acquires the lock again in this method. So no need to keep holding the lock when invoking HMS RPCs. This patch removes holding the lock in alterTableOrViewRename(). Tests - Added e2e test for concurrent rename operations. - Also added some rename operations in test_concurrent_ddls.py Change-Id: Ie5f443b1e167d96024b717ce70ca542d7930cb4b --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_concurrent_ddls.py M tests/metadata/test_ddl.py 3 files changed, 65 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/22789/1 -- To view, visit http://gerrit.cloudera.org:8080/22789 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie5f443b1e167d96024b717ce70ca542d7930cb4b Gerrit-Change-Number: 22789 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
