I filed HIVE-21409 for this issue. It looks like the registerJars method in SessionState is still using the thread context class loader instead of grabbing the one from SessionState. I'm not sure how it happens but it looks like if you have a bunch of parallel sessions from something like Oozie right after Hive starts the class path can get really polluted quickly. I need to go look at the mechanism behind the "delete jars" command as running that on a brand new session seems to wipe all the extra stuff out of the class loader.
Thanks -----Original Message----- From: Gopal Vijayaraghavan <gop...@apache.org> Sent: Wednesday, March 6, 2019 3:01 PM To: dev@hive.apache.org Subject: Re: Custom UDF Loses Depenencies > When we register a jar on the Hive console. Hive creates a fresh URL > classloader which includes the path of the current jar to be registered and > all the jar paths of the parent classloader. The parent classlaoder is the > current ThreadContextClassLoader. Once the URLClassloader is created Hive > sets ... That looks like the root-cause of a different class of issues. Since the fresh classloader is picking up the URLs, the URLs keep growing. Worse, the Hadoop classes loaded from shims are coming off the thread loader & being reloaded a million times. Cheers, Gopal
