[
https://issues.apache.org/jira/browse/HIVE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778019#comment-13778019
]
Alexander Pivovarov commented on HIVE-4501:
-------------------------------------------
if cache is enabled then only one FileSystem per query is not closed
But if cache is disabled then 4 FileSystems are not closed - as a result 4
lease checker threads leak per query
How disabling FileSystem cache can help with closing FileSystem?
if we write smth to hdfs then Lease Checker thread is created. To stop the
thread File System should be closed.
The only question is where.
As we already know calling close() at he end of TUGIContainingProcessor.process
is too early.
So, where should it be called?
> HS2 memory leak - FileSystem objects in FileSystem.CACHE
> --------------------------------------------------------
>
> Key: HIVE-4501
> URL: https://issues.apache.org/jira/browse/HIVE-4501
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 0.11.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Priority: Critical
> Attachments: HIVE-4501.1.patch
>
>
> org.apache.hadoop.fs.FileSystem objects are getting accumulated in
> FileSystem.CACHE, with HS2 in unsecure mode.
> As a workaround, it is possible to set fs.hdfs.impl.disable.cache and
> fs.file.impl.disable.cache to true.
> Users should not have to bother with this extra configuration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
