Is "jar tf /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" listing for example the "org/apache/flink/client/CliFrontend" class?
On Wed, Nov 11, 2015 at 12:09 PM, Maximilian Michels <[email protected]> wrote: > Hi Camelia, > > Flink 0.9.X supports Java 6. So this can't be the issue. > > Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I > was able to start the command-line interface, job manager and task managers. > > java version "1.6.0_36" > OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04) > OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode) > > > I think the error must be caused by your NFS setup. Let's start with > ./bin/flink. For instance, can you access > "/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" from the > machine where you run ./bin/flink? > > Best, > Max > > > On Wed, Nov 11, 2015 at 10:41 AM, Camelia Elena Ciolac < > [email protected]> wrote: > >> Hello, >> >> As promised, I come back with debugging details. >> So: >> >> ******* In start-cluster.sh , the following echo's >> >> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~" >> echo $HOSTLIST >> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~" >> echo $FLINK_BIN_DIR >> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~" >> >> # cluster mode, bring up job manager locally and a task manager on every >> slave host >> "$FLINK_BIN_DIR"/jobmanager.sh start cluster batch >> >> ==> gave: >> >> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster/flink-0.9.1/conf/slaves >> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster/flink-0.9.1/bin >> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> ******* In jobmanager.sh , the following echo's >> >> echo "Starting Job Manager" >> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~" >> pwd >> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo ${FLINK_ENV_JAVA_OPTS} >> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo $FLINK_JM_CLASSPATH >> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo $INTERNAL_HADOOP_CLASSPATHS >> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~" >> echo $FLINK_CONF_DIR >> echo "~~~~~~~~~~~~~~~~~~~~" >> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" >> -classpath "`manglePathList >> "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" >> org.apache.flink.runtime.jobmanager.JobManager --configDir >> "$FLINK_CONF_DIR" --executionMode $EXECUTIONMODE --streamingMode >> "$STREAMINGMODE" > "$out" 2>&1 < /dev/null & >> >> >> >> ==> gave: >> >> jobmanager ~~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster >> jobmanager ~~~~~~~~~~~~~~~~~~~~~ >> >> jobmanager ~~~~~~~~~~~~~~~~~~~~~ >> >> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar >> jobmanager ~~~~~~~~~~~~~~~~~~~~~ >> :: >> jobmanager ~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster/flink-0.9.1/conf >> ~~~~~~~~~~~~~~~~~~~~ >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/runtime/jobmanager/JobManager >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.runtime.jobmanager.JobManager >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: >> org.apache.flink.runtime.jobmanager.JobManager. Program will exit. >> >> >> >> ******* In taskmanager.sh, the following echo's >> >> echo Starting task manager on host $HOSTNAME >> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~" >> pwd >> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo ${FLINK_ENV_JAVA_OPTS} >> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo $FLINK_JM_CLASSPATH >> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~" >> echo $INTERNAL_HADOOP_CLASSPATHS >> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~" >> echo $FLINK_CONF_DIR >> echo "~~~~~~~~~~~~~~~~~~~~" >> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" >> -classpath "`manglePathList >> "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" >> org.apache.flink.runtime.taskmanager.TaskManager --configDir >> "$FLINK_CONF_DIR" --streamingMode "$STREAMINGMODE" > "$out" 2>&1 < >> /dev/null & >> >> ==> gave: >> >> taskmanager ~~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster >> taskmanager ~~~~~~~~~~~~~~~~~~~~~ >> >> taskmanager ~~~~~~~~~~~~~~~~~~~~~ >> >> taskmanager ~~~~~~~~~~~~~~~~~~~~~ >> :: >> taskmanager ~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster/flink-0.9.1/conf >> ~~~~~~~~~~~~~~~~~~~~ >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/runtime/taskmanager/TaskManager >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.runtime.taskmanager.TaskManager >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: >> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit. >> >> >> >> ******* In flink , the following echo's >> >> echo "flink ~~~~~~~~~~~~~~~~~~~~~" >> pwd >> echo "flink ~~~~~~~~~~~~~~~~~~~~~" >> echo $CC_CLASSPATH >> echo "flink ~~~~~~~~~~~~~~~~~~~~~" >> echo $INTERNAL_HADOOP_CLASSPATHS >> echo "~~~~~~~~~~~~~~~~~~~~" >> $JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList >> "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" >> org.apache.flink.client.CliFrontend "$@" >> >> >> ==> gave: >> >> flink ~~~~~~~~~~~~~~~~~~~~~ >> /users/camelia/thecluster >> flink ~~~~~~~~~~~~~~~~~~~~~ >> >> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar >> flink ~~~~~~~~~~~~~~~~~~~~~ >> :: >> ~~~~~~~~~~~~~~~~~~~~ >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/client/CliFrontend >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.client.CliFrontend >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: org.apache.flink.client.CliFrontend. >> Program will exit. >> >> >> I hope that now you have a better image of the issue and I hope you can >> provide some guidance to solve it. >> I mention that I use the simplest scenario possible, just 2 nodes (1 >> master and 1 slave) without Hadoop for this test. >> Also, I have a hostname command inside the script that shows me that I'm >> on the master node when running these scripts. >> >> Thank you very much! >> Camelia >> >> >> ------------------------------ >> *From:* Camelia Elena Ciolac >> *Sent:* Wednesday, November 11, 2015 9:01 AM >> *To:* [email protected] >> *Subject:* RE: Cluster installation gives java.lang.NoClassDefFoundError >> for everything >> >> Good morning, >> >> Thank you Stephan! >> >> I keep on testing and in the meantime I'm wondering if the Java version >> on the cluster may be part of the issue: >> java version "1.6.0_36" >> OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64) >> OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode) >> ? >> >> The cluster has shared network file system, so no problem in this aspect. >> I keep on trying and I've put some echo's for debugging in the Flink >> start scripts. >> >> Best regards, >> Camelia >> >> >> >> ------------------------------ >> *From:* Camelia Elena Ciolac >> *Sent:* Monday, November 09, 2015 2:32 PM >> *To:* [email protected] >> *Subject:* Cluster installation gives java.lang.NoClassDefFoundError for >> everything >> >> Hello, >> >> I am configuring Flink to run on a cluster with NFS. >> I have the Flink 0.9.1 distribution in some path in NFS and I added that >> path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib >> folder to $PATH. >> I have the slaves file and the yaml file configured correctly with the >> nodes involved. >> >> Still, when I run the following command I get the following errors: >> >> $FLINK_HOME/bin/start-cluster.sh >> >> (tried also >> cd $FLINK_HOME >> ./bin/start-cluster.sh) >> >> --> in /log/flink-theuser-jobmanager-thenode.out >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/runtime/jobmanager/JobManager >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.runtime.jobmanager.JobManager >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: >> org.apache.flink.runtime.jobmanager.JobManager. Program will exit. >> >> -->in /log/flink-theuser-taskmanager-theslavenode.out >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/runtime/taskmanager/TaskManager >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.runtime.taskmanager.TaskManager >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: >> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit. >> >> >> And lastly, in my script I have the run of a job which gives: >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/flink/client/CliFrontend >> Caused by: java.lang.ClassNotFoundException: >> org.apache.flink.client.CliFrontend >> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:323) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:268) >> Could not find the main class: org.apache.flink.client.CliFrontend. >> Program will exit. >> >> What could be the cause and which is the solution? >> >> Looking forward to your answer as soon as possible. >> Many thanks, >> Camelia >> >> >
