[
https://issues.apache.org/jira/browse/HIVE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13835301#comment-13835301
]
Harel Ben Attia commented on HIVE-5853:
---------------------------------------
Didn't have the opportunity to try HS2 yet, but in our case the connection leak
was happening even without any client concurrency. Just running a python script
multiple times, each performing one simple query through hive-thrift. At first
I thought it might be related to improper connection closing in the script
itself, but that wasn't the case, as our production system uses the Java
client, and the same result happens there as well.
I'd be glad to try and help - just tell me what you need.
> Hive Lock Manager leaks zookeeper connections
> ---------------------------------------------
>
> Key: HIVE-5853
> URL: https://issues.apache.org/jira/browse/HIVE-5853
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Harel Ben Attia
>
> Hive 0.10 leaks zookeeper connections from ZooKeeperHiveLockManager.
> HIVE-3723 describes a similar issue for cases of semantic errors and
> failures, but we're experiencing a consistent connection leak per query (even
> simple successful queries like "select * from dual").
> Workaround: When turning off hive.support.concurrency, everything works fine
> - no leak (obviously, since the lock manager is not used).
> Details:
> OS: CentOS 5.9
> Hive version: hive-server-0.10.0+67-1.cdh4.2.0.p0.10.el5 and
> hive-0.10.0+198-1.cdh4.4.0.p0.15.el5
> Hadoop version: CDH4.2
> Namenode uses HA. Hive's zookeeper configuration uses the NN zookeeper.
> The problem occurs both when using the python thrift API, and the java thrift
> API.
> The leak happens even when we're running repeated "select * from dual"
> queries. We've checked the zookeeper connections using "netstat -n | grep
> 2181 | grep ESTAB | wc -l".
> Eventually, the connection from the client reach the max connections per
> client limit in ZK, causing new queries to get stuck and never return.
> We'll gladly provide more information if needed.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
