I couldn't spot any error in what you tried to do. Does the
job-submission succeed if you submit the jar through the command-line
client?
Can you share the project, or a minimal reproducing version?
On 25.04.2018 00:41, kedar mhaswade wrote:
I am trying to get gradoop_demo
<https://github.com/dbs-leipzig/gradoop_demo> (a gradoop based graph
visualization app) working on Flink with *Remote* Execution Environment.
This app, which is based on Gradoop, submits a job to the
/preconfigured/ execution environment, collects the results and sends
it to the UI for rendering.
When the execution environment is configured to be a LocalEnvironment
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/api/java/LocalEnvironment.html>,
everything works fine. But when I start a cluster (using
<flink-install-path>/bin/start-cluster.sh), get the Job Manager
endpoint (e.g. localhost:6123) and configure a RemoteEnvironment
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/api/java/ExecutionEnvironment.html#createRemoteEnvironment-java.lang.String-int-org.apache.flink.configuration.Configuration-java.lang.String...-> and
use that environment to run the job, I get exceptions [1].
Based on the class loading doc
<https://ci.apache.org/projects/flink/flink-docs-master/monitoring/debugging_classloading.html>,
I copied the gradoop classes
(gradoop-flink-0.3.3-SNAPSHOT.jar, gradoop-common-0.3.3-SNAPSHOT.jar)
to the <flink-install-path>/lib folder (hoping that that way those
classes will be available to all the executors in the cluster). I have
ensured that the class that Flink fails to load is in fact available
in the Gradoop jars that I copied to the /lib folder.
I have tried using the RemoteEnvironment method with jarFiles argument
where the passed JAR file is a fat jar containing everything (in which
case there is no Gradoop JAR file in /lib folder).
So, my questions are:
1) How can I use RemoteEnvironment?
2) Is there any other way of doing this /programmatically? /(That
means I can't do flink run since I am interested in the job execution
result as a blocking call -- which means ideally I don't want to use
the submit RESTful API as well). I just want RemoteEnvironment to work
as well as LocalEnvironment.
Regards,
Kedar
[1]
2018-04-24 15:16:02,823 ERROR
org.apache.flink.runtime.jobmanager.JobManager - Failed to
submit job 0c987c8704f8b7eb4d7d38efcb3d708d (Flink Java Job at Tue Apr
24 15:15:59 PDT 2018)
java.lang.NoClassDefFoundError: Could not initialize class
*org.gradoop.common.model.impl.id.GradoopId*
at java.io.ObjectStreamClass.hasStaticInitializer(Native Method)
at
java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1887)
at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:79)
at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:263)
at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at
java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:260)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:682)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1876)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1745)
at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1710)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1550)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427)
at java.util.HashSet.readObject(HashSet.java:341)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2278)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2202)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427)
at
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290)....
