[
https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296771#comment-15296771
]
Thejas M Nair commented on HIVE-13749:
--------------------------------------
This FileSystem.CACHE leak is an issue that shows up in several places across
projects. Its not a cache in the traditional sense, that it doesn't have an
eviction policy.
The peculiar thing about this cache is that the underlying Map has a key that
contains UGI object, and the key comparison uses object equality (== operator)
instead of .equals method for comparison. This is apparently by design for some
security aspects.
So you have one entry in the cache for every UGI object.
See the final patch in HIVE-3098 for example. The usual way to solve this is to
call FileSystem.closeAllForUGI once the UGI use is over.
Other related leak jiras - HIVE-4501, HIVE-9234 .
You might want to check if this is happening in latest trunk as well. There
might be other patches that have fixed the problem.
> Memory leak in Hive Metastore
> -----------------------------
>
> Key: HIVE-13749
> URL: https://issues.apache.org/jira/browse/HIVE-13749
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 1.1.0
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Attachments: Top_Consumers7.html
>
>
> Looking a heap dump of 10GB, a large number of Configuration objects(> 66k
> instances) are being retained. These objects along with its retained set is
> occupying about 95% of the heap space. This leads to HMS crashes every few
> days.
> I will attach an exported snapshot from the eclipse MAT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
