[
https://issues.apache.org/jira/browse/HIVE-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730122#comment-14730122
]
Thejas M Nair commented on HIVE-11408:
--------------------------------------
Change
{code}
// HIVE-11408, HADOOP-10513
// Hadoop's ReflectionUtils caches constructors for the classes it
instantiated.
// In UDFs, this can result in classloaders not getting GCed for a temporary
function,
// resulting in a PermGen leak when used extensively from HiveServer2
{code}
to
{code}
// Hadoop's ReflectionUtils caches constructors for the classes it
instantiated.
// In UDFs, this can result in classloaders not getting GCed for a temporary
function,
// resulting in a PermGen leak when used extensively from HiveServer2. See
profiler snapshot in HADOOP-10513
{code}
?
> HiveServer2 is leaking ClassLoaders when add jar / temporary functions are
> used
> -------------------------------------------------------------------------------
>
> Key: HIVE-11408
> URL: https://issues.apache.org/jira/browse/HIVE-11408
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.1.1
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Attachments: HIVE-11408.1.patch
>
>
> I'm able to reproduce with 0.14. I'm yet to see if HIVE-10453 fixes the issue
> (since it's on top of a larger patch: HIVE-2573 that was added in 1.2).
> Basically, add jar creates a new classloader for loading the classes from the
> new jar and adds the new classloader to the SessionState object of user's
> session, making the older one its parent. Creating a temporary function uses
> the new classloader to load the class used for the function. On closing a
> session, although there is code to close the classloader for the session, I'm
> not seeing the new classloader getting GCed and from the heapdump I can see
> it holds on to the temporary function's class that should have gone away
> after the session close.
> Steps to reproduce:
> 1.
> {code}
> jdbc:hive2://localhost:10000/> add jar hdfs:///tmp/audf.jar;
> {code}
> 2.
> Use a profiler (I'm using yourkit) to verify that a new URLClassLoader was
> added.
> 3.
> {code}
> jdbc:hive2://localhost:10000/> CREATE TEMPORARY FUNCTION funcA AS
> 'org.gumashta.udf.AUDF';
> {code}
> 4.
> Close the jdbc session.
> 5.
> Take the memory snapshot and verify that the new URLClassLoader is indeed
> there and is holding onto the class it loaded (org.gumashta.udf.AUDF) for the
> session which we already closed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
