[
https://issues.apache.org/jira/browse/HIVE-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255487#comment-14255487
]
Na Yang commented on HIVE-9119:
-------------------------------
According to a ZooKeeper test that I did, increasing the number of ZooKeeper
client instances does not help improving the ZooKeeper operation requests
performance in a multi-thread environment. The major load is on the ZooKeeper
server side. Therefore, a singleton ZooKeeper client is a good choice for the
ZookeeperHiveLockManager implementation.
A zookeeper client/connection pool is not needed in this case. In addition, if
a zookeeper client/connection pool is used and the pool side is set too big by
the user, that can cause overhead on zookeeper server and even make the
ZooKeeperHiveLockManager not workable if the zookeeper client number exceeds
the number that the zookeeper server could handle.
> ZooKeeperHiveLockManager does not use zookeeper in the proper way
> -----------------------------------------------------------------
>
> Key: HIVE-9119
> URL: https://issues.apache.org/jira/browse/HIVE-9119
> Project: Hive
> Issue Type: Improvement
> Components: Locking
> Affects Versions: 0.13.0, 0.14.0, 0.13.1
> Reporter: Na Yang
> Assignee: Na Yang
> Attachments: HIVE-9119.1.patch
>
>
> ZooKeeperHiveLockManager does not use zookeeper in the proper way.
> Currently a new zookeeper client instance is created for each
> getlock/releaselock query which sometimes causes the number of open
> connections between
> HiveServer2 and ZooKeeper exceed the max connection number that zookeeper
> server allows.
> To use zookeeper as a distributed lock, there is no need to create a new
> zookeeper instance for every getlock try. A single zookeeper instance could
> be reused and shared by ZooKeeperHiveLockManagers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
