I’m not saying we can’t change that code to use the context class loader. I’m just not sure whether this might break other things.
Best, Aljoscha > On 2. Sep 2019, at 11:24, Jan Lukavský <je...@seznam.cz> wrote: > > Essentially, the class loader of Flink should be present in parent hierarchy > of context class loader. If FlinkUserCodeClassLoader doesn't use context > class loader, then it is actually impossible to use a hierarchy like this: > > system class loader -> application class loader -> user-defined class loader > (defines some UDFs to be used in flink program) > > Flink now uses the application class loader, and so the UDFs fail to > deserialize on local flink, because the user-defined class loader is > bypassed. Moreover, there is no way to add additional classpath elements for > LocalEnvironment (as opposed to RemoteEnvironment). I'm able to hack this by > calling addURL method on the application class loader (which is terribly > hackish), but that works only on JDK <= 8. No sensible workaround is > available for JDK >= 9. > > Alternative solution would be to enable adding jars to class loader when > using LocalEnvironment, but that looks a little odd. > > Jan > > On 9/2/19 11:02 AM, Aljoscha Krettek wrote: >> Hi, >> >> I actually don’t know whether that change would be ok. >> FlinkUserCodeClassLoader has taken >> FlinkUserCodeClassLoader.class.getClassLoader() as the parent ClassLoader >> before my change. See: >> https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoader.java >> >> <https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoader.java>. >> >> I have the feeling that this might be on purpose because we want to have the >> ClassLoader of the Flink Framework components be the parent ClassLoader, but >> I could be wrong. Maybe Stephan would be most appropriate for answering this. >> >> Best, >> Aljoscha >> >>> On 30. Aug 2019, at 16:28, Till Rohrmann <trohrm...@apache.org> wrote: >>> >>> Hi Jan, >>> >>> this looks to me like a bug for which you could create a JIRA and PR to fix >>> it. Just to make sure, I've pulled in Aljoscha who is the author of this >>> change to check with him whether we are forgetting something. >>> >>> Cheers, >>> Till >>> >>> On Fri, Aug 30, 2019 at 3:44 PM Jan Lukavský <je...@seznam.cz >>> <mailto:je...@seznam.cz>> wrote: >>> Hi, >>> >>> I have come across an issue with classloading in Flink's MiniCluster. >>> The issue is that when I run local flink job from a thread, that has a >>> non-default context classloader (for whatever reason), this classloader >>> is not taken into account when classloading user defined functions. This >>> is due to [1]. Is this behavior intentional, or can I file a JIRA and >>> use Thread.currentThread.getContextClassLoader() there? I have validated >>> that it fixes issues I'm facing. >>> >>> Jan >>> >>> [1] >>> https://github.com/apache/flink/blob/ce557839d762b5f1ec92aa1885fd3d2ae33d0d0b/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L280 >>> >>> <https://github.com/apache/flink/blob/ce557839d762b5f1ec92aa1885fd3d2ae33d0d0b/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L280> >>> >>
