[
https://issues.apache.org/jira/browse/HIVE-9234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14300915#comment-14300915
]
Thejas M Nair commented on HIVE-9234:
-------------------------------------
The leak was in FileSystem.CACHE, it would keep growing. The Key in the CACHE
includes UGI object. To clear the entries in CACHE corresponding to the UGI
object, closeAllForUGI should be called once the UGI is no longer going to be
used.
The fix is in changing the sequence - to do
ShimLoader.getHadoopShims().closeAllForUGI(sessionUgi) as the final step of
HiveSessionImplwithUGI.close, after super.close(). super.close() was
indirectly adding another entry into the CACHE.
> HiveServer2 leaks FileSystem objects in FileSystem.CACHE
> --------------------------------------------------------
>
> Key: HIVE-9234
> URL: https://issues.apache.org/jira/browse/HIVE-9234
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 0.12.0, 0.13.0, 0.12.1, 0.14.0, 0.13.1
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Fix For: 0.15.0, 1.0.0
>
> Attachments: HIVE-9234.1.patch, HIVE-9234.2.patch, HIVE-9234.2.patch,
> HIVE-9234.branch-14.patch
>
>
> Running over extended period (48+ hrs), we've noticed HiveServer2 leaking
> FileSystem objects in FileSystem.CACHE. Linked jiras were previous attempts
> to fix it, but the issue still seems to be there. A workaround is to disable
> the caching (by setting {{fs.hdfs.impl.disable.cache}} and
> {{fs.file.impl.disable.cache}} to {{true}}), but creating new FileSystem
> objects is expensive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
