[
https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304586#comment-15304586
]
Thejas M Nair commented on HIVE-13749:
--------------------------------------
bq. Since disabling the cache entirely causes no functional regression,
wouldn't adding a call to delete elements from this cache would be fine too,
functionally
No, that method is closeAllForUGI, not deleteAllForUGIFromCache, ie it closes
the FileSystem instances. In case of metastore being used in embedded mode
(-hiveconf hive.metastore.uris=' ') , if there are references to the FileSystem
object in other parts of the code, they will be suddenly closed instances.
[~aihuaxu] The FileSystem.CACHE keeps the references to the FileSystem objects
around even if most of them aren't being used and GC will not free them as a
result.
The changes in HIVE-3098 should be addressing this, we need to understand why
it wouldn't do that before adding yet another call to closeAllForUGI.
We haven't seen this OOM in metastore with hive 1.2.0 users, except for an
issue when ACID was enabled, which [~wzheng] worked on. So one thing to analyze
is if any plugins are creating new UGI object and that is causing this OOM.
[~daijy] has some experience with dealing with this type of leak. He can also
advise on this.
> Memory leak in Hive Metastore
> -----------------------------
>
> Key: HIVE-13749
> URL: https://issues.apache.org/jira/browse/HIVE-13749
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 1.1.0
> Reporter: Naveen Gangam
> Assignee: Naveen Gangam
> Attachments: HIVE-13749.patch, Top_Consumers7.html
>
>
> Looking a heap dump of 10GB, a large number of Configuration objects(> 66k
> instances) are being retained. These objects along with its retained set is
> occupying about 95% of the heap space. This leads to HMS crashes every few
> days.
> I will attach an exported snapshot from the eclipse MAT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
