[
https://issues.apache.org/jira/browse/FLINK-6767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt updated FLINK-6767:
------------------------
Description:
There is a bug in Flink 1.2.1 that results in a "cannot load user class"
exception even when the class is available in the current class loader of the
thread running the job. The problem arises when you execute a Flink job on a
local environment inside an Ignite 2.0 node. This happens on possibly all other
versions of Flink and Ignite.
This bug was discussed in [1], and a fix was proposed in [2].
In summary, the fix requires replacing line 298 in
*BlobLibraryCacheManager.java* [3] for:
{code:java}
this.classLoader = new FlinkUserCodeClassLoader(libraryURLs,
Thread.currentThread().getContextClassLoader());
{code}
A repository with a complete test case reproducing the error is found in [4].
The idea behind the code is being able to run Flink jobs in a collocated way
(ie, on the node where the data is stored), minimizing network traffic and thus
improving the performance.
The README file contains details on how to run it and the resulting exception:
{code}
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user
class: com.test.Source
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
{code}
[1]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-td12799.html
[2]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-tp12799p13376.html
[3]
https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L298
[4] https://github.com/Dromit/FlinkTest
was:
There is a bug in Flink 1.2.1 that results in a "cannot load user class"
exception even when the class is available in the current class loader of the
thread running the job. The problem arises when you execute a Flink job on a
local environment inside an Ignite 2.0 node. This happens on possibly all other
versions of Flink and Ignite.
This bug was discussed in [1], and a fix was proposed in [2].
In summary, the fix requires replacing line 298 in
*BlobLibraryCacheManager.java* [3] for:
{code:java}
this.classLoader = new FlinkUserCodeClassLoader(libraryURLs,
Thread.currentThread().getContextClassLoader());
{code}
A repository with a complete test case reproducing the error is found in [4].
The idea behind the code is being able to run Flink jobs in a collocated way
(ie, on the node where the data is stored), minimizing network traffic and thus
improving the performance.
The README file contains details on how to run it and the resulting exception.
{code}
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user
class: com.test.Source
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
{code}
[1]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-td12799.html
[2]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-tp12799p13376.html
[3]
https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L298
[4] https://github.com/Dromit/FlinkTest
> Cannot load user class on local environment
> -------------------------------------------
>
> Key: FLINK-6767
> URL: https://issues.apache.org/jira/browse/FLINK-6767
> Project: Flink
> Issue Type: Bug
> Components: Local Runtime
> Affects Versions: 1.2.1
> Environment: Flink 1.2.1 running on local environment on a Ignite 2.0
> node.
> Reporter: Matt
> Priority: Critical
> Labels: CollocatedComputation, Flink, Ignite, LocalEvironment
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> There is a bug in Flink 1.2.1 that results in a "cannot load user class"
> exception even when the class is available in the current class loader of the
> thread running the job. The problem arises when you execute a Flink job on a
> local environment inside an Ignite 2.0 node. This happens on possibly all
> other versions of Flink and Ignite.
> This bug was discussed in [1], and a fix was proposed in [2].
> In summary, the fix requires replacing line 298 in
> *BlobLibraryCacheManager.java* [3] for:
> {code:java}
> this.classLoader = new FlinkUserCodeClassLoader(libraryURLs,
> Thread.currentThread().getContextClassLoader());
> {code}
> A repository with a complete test case reproducing the error is found in [4].
> The idea behind the code is being able to run Flink jobs in a collocated way
> (ie, on the node where the data is stored), minimizing network traffic and
> thus improving the performance.
> The README file contains details on how to run it and the resulting exception:
> {code}
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load
> user class: com.test.Source
> ClassLoader info: URL ClassLoader:
> Class not resolvable through given classloader.
> {code}
> [1]
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-td12799.html
> [2]
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BUG-Cannot-Load-User-Class-on-Local-Environment-tp12799p13376.html
> [3]
> https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L298
> [4] https://github.com/Dromit/FlinkTest
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
