I am trying to get gradoop_demo <https://github.com/dbs-leipzig/gradoop_demo> (a gradoop based graph visualization app) working on Flink with *Remote* Execution Environment.
This app, which is based on Gradoop, submits a job to the *preconfigured* execution environment, collects the results and sends it to the UI for rendering. When the execution environment is configured to be a LocalEnvironment <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/api/java/LocalEnvironment.html>, everything works fine. But when I start a cluster (using < flink-install-path>/bin/start-cluster.sh), get the Job Manager endpoint (e.g. localhost:6123) and configure a RemoteEnvironment <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/api/java/ExecutionEnvironment.html#createRemoteEnvironment-java.lang.String-int-org.apache.flink.configuration.Configuration-java.lang.String...-> and use that environment to run the job, I get exceptions [1]. Based on the class loading doc <https://ci.apache.org/projects/flink/flink-docs-master/monitoring/debugging_classloading.html>, I copied the gradoop classes ( gradoop-flink-0.3.3-SNAPSHOT.jar, gradoop-common-0.3.3-SNAPSHOT.jar) to the <flink-install-path>/lib folder (hoping that that way those classes will be available to all the executors in the cluster). I have ensured that the class that Flink fails to load is in fact available in the Gradoop jars that I copied to the /lib folder. I have tried using the RemoteEnvironment method with jarFiles argument where the passed JAR file is a fat jar containing everything (in which case there is no Gradoop JAR file in /lib folder). So, my questions are: 1) How can I use RemoteEnvironment? 2) Is there any other way of doing this *programmatically? *(That means I can't do flink run since I am interested in the job execution result as a blocking call -- which means ideally I don't want to use the submit RESTful API as well). I just want RemoteEnvironment to work as well as LocalEnvironment. Regards, Kedar [1] 2018-04-24 15:16:02,823 ERROR org.apache.flink.runtime.jobmanager.JobManager - Failed to submit job 0c987c8704f8b7eb4d7d38efcb3d708d (Flink Java Job at Tue Apr 24 15:15:59 PDT 2018) java.lang.NoClassDefFoundError: Could not initialize class *org.gradoop.common.model.impl.id.GradoopId* at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1887) at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:79) at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:263) at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:261) at java.security.AccessController.doPrivileged(Native Method) at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:260) at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:682) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1876) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1745) at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1710) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1550) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427) at java.util.HashSet.readObject(HashSet.java:341) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2278) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2202) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290)....
