i start the yarn-session.sh with sudo and than the flink run command with sudo, i get the following exception:
cloudera@quickstart bin]$ sudo ./flink run /home/cloudera/Desktop/ma-flink.jar log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. org.apache.flink.client.program.ProgramInvocationException: Failed to resolve JobManager at org.apache.flink.client.program.Client.run(Client.java:378) at org.apache.flink.client.program.Client.run(Client.java:355) at org.apache.flink.client.program.Client.run(Client.java:348) at org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) at mgm.tp.bigdata.ma_flink.FlinkMain.main(FlinkMain.java:70) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353) at org.apache.flink.client.program.Client.run(Client.java:315) at org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:880) at org.apache.flink.client.CliFrontend.main(CliFrontend.java:922) Caused by: java.io.IOException: JobManager at akka.tcp:// flink@127.0.0.1:6123/user/jobmanager not reachable. Please make sure that the JobManager is running and its port is reachable. at org.apache.flink.runtime.jobmanager.JobManager$.getJobManagerRemoteReference(JobManager.scala:1198) at org.apache.flink.runtime.jobmanager.JobManager$.getJobManagerRemoteReference(JobManager.scala:1222) at org.apache.flink.runtime.jobmanager.JobManager$.getJobManagerRemoteReference(JobManager.scala:1240) at org.apache.flink.runtime.jobmanager.JobManager.getJobManagerRemoteReference(JobManager.scala) at org.apache.flink.client.program.Client.run(Client.java:375) ... 15 more Caused by: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://flink@127.0.0.1:6123/), Path(/user/jobmanager)] at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65) at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74) at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110) at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73) at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248) at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267) at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:508) at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:541) at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:531) at akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87) at akka.remote.EndpointWriter.postStop(Endpoint.scala:561) at akka.actor.Actor$class.aroundPostStop(Actor.scala:475) at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:415) at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210) at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172) at akka.actor.ActorCell.terminate(ActorCell.scala:369) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:279) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.Mailbox.exec(Mailbox.scala:231) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) the FlinkMain.java: 70 is: env.execute("KMeans Flink"); 2015-06-04 17:17 GMT+02:00 Pa Rö <paul.roewer1...@googlemail.com>: > i try this: > > [cloudera@quickstart bin]$ sudo su yarn > bash-4.1$ hadoop fs -chmod 777 /user/cloudera/outputs > chmod: changing permissions of '/user/cloudera/outputs': Permission > denied. user=yarn is not the owner of inode=outputs > bash-4.1$ hadoop fs -chmod 777 /user/cloudera/inputs > chmod: changing permissions of '/user/cloudera/inputs': Permission denied. > user=yarn is not the owner of inode=inputs > bash-4.1$ exit > exit > [cloudera@quickstart bin]$ sudo ./flink run > /home/cloudera/Desktop/ma-flink.jar > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > more info. > Found YARN properties file > /home/cloudera/Desktop/flink-0.9-SNAPSHOT/bin/../conf/.yarn-properties > Using JobManager address from YARN properties quickstart.cloudera/ > 127.0.0.1:53874 > org.apache.flink.client.program.ProgramInvocationException: The program > execution failed: Failed to submit job 2f46ef5dff4ecf5552b3477ed1c6f4b9 > (KMeans Flink) > at org.apache.flink.client.program.Client.run(Client.java:412) > at org.apache.flink.client.program.Client.run(Client.java:355) > at org.apache.flink.client.program.Client.run(Client.java:348) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) > at mgm.tp.bigdata.ma_flink.FlinkMain.main(FlinkMain.java:70) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353) > at org.apache.flink.client.program.Client.run(Client.java:315) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:880) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:922) > Caused by: org.apache.flink.runtime.client.JobExecutionException: Failed > to submit job 2f46ef5dff4ecf5552b3477ed1c6f4b9 (KMeans Flink) > at org.apache.flink.runtime.jobmanager.JobManager.org > $apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:595) > at > org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:192) > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) > at > scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) > at > org.apache.flink.yarn.ApplicationMasterActor$$anonfun$receiveYarnMessages$1.applyOrElse(ApplicationMasterActor.scala:99) > at scala.PartialFunction$OrElse.apply(PartialFunction.scala:162) > at > org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:36) > at > org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:29) > at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) > at > org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:29) > at akka.actor.Actor$class.aroundReceive(Actor.scala:465) > at > org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:94) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) > at akka.actor.ActorCell.invoke(ActorCell.scala:487) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) > at akka.dispatch.Mailbox.run(Mailbox.scala:221) > at akka.dispatch.Mailbox.exec(Mailbox.scala:231) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > Caused by: org.apache.flink.runtime.JobException: Creating the input > splits caused an error: File /user/cloudera/inputs does not exist or the > user running Flink ('yarn') has insufficient permissions to access it. > at > org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:162) > at > org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:471) > at org.apache.flink.runtime.jobmanager.JobManager.org > $apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:535) > ... 21 more > Caused by: java.io.FileNotFoundException: File /user/cloudera/inputs does > not exist or the user running Flink ('yarn') has insufficient permissions > to access it. > at > org.apache.flink.core.fs.local.LocalFileSystem.getFileStatus(LocalFileSystem.java:106) > at > org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:390) > at > org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:51) > at > org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:146) > ... 23 more > > > 2015-06-04 17:15 GMT+02:00 Robert Metzger <rmetz...@apache.org>: > >> As the output of the "hadoop" tool indicates, it expects two arguments, >> you only passed one (777). >> The second argument it is expecting is the path to the file you want to >> change. >> >> In your case, it is: >> hadoop fs -chmod 777 /user/cloudera/outputs >> >> >> The reason why >> hadoop fs -chmod 777 * >> does not work is the following: the * is evaluated by your local bash and >> expanded to the files which are present in your current, local directory. >> The bash expansion is not able to expand to the files in HDFS. >> >> >> On Thu, Jun 4, 2015 at 5:08 PM, Pa Rö <paul.roewer1...@googlemail.com> >> wrote: >> >>> [cloudera@quickstart bin]$ sudo su yarn >>> bash-4.1$ hadoop fs -chmod 777 >>> -chmod: Not enough arguments: expected 2 but got 1 >>> Usage: hadoop fs [generic options] -chmod [-R] <MODE[,MODE]... | >>> OCTALMODE> PATH... >>> bash-4.1$ >>> >>> you understand? >>> >>> 2015-06-04 17:04 GMT+02:00 Robert Metzger <rmetz...@apache.org>: >>> >>>> It looks like the user "yarn" which is running Flink doesn't have >>>> permission to access the files. >>>> >>>> Can you do "sudo su yarn" to become the "yarn" user. Then, you can do >>>> "hadoop fs -chmod 777" to make the files accessible for everyone. >>>> >>>> >>>> On Thu, Jun 4, 2015 at 4:59 PM, Pa Rö <paul.roewer1...@googlemail.com> >>>> wrote: >>>> >>>>> okay, it's work, i get a exception: >>>>> >>>>> [cloudera@quickstart Desktop]$ cd flink-0.9-SNAPSHOT/bin/ >>>>> [cloudera@quickstart bin]$ flink run >>>>> /home/cloudera/Desktop/ma-flink.jar >>>>> bash: flink: command not found >>>>> [cloudera@quickstart bin]$ ./flink run >>>>> /home/cloudera/Desktop/ma-flink.jar >>>>> log4j:WARN No appenders could be found for logger >>>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). >>>>> log4j:WARN Please initialize the log4j system properly. >>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig >>>>> for more info. >>>>> Found YARN properties file >>>>> /home/cloudera/Desktop/flink-0.9-SNAPSHOT/bin/../conf/.yarn-properties >>>>> Using JobManager address from YARN properties quickstart.cloudera/ >>>>> 127.0.0.1:53874 >>>>> java.io.IOException: Mkdirs failed to create /user/cloudera/outputs >>>>> at >>>>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:438) >>>>> at >>>>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424) >>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:905) >>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:886) >>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:783) >>>>> at >>>>> mgm.tp.bigdata.ma_commons.commons.Seeding.randomSeeding(Seeding.java:21) >>>>> at >>>>> mgm.tp.bigdata.ma_flink.FlinkMain.getCentroidDataSet(FlinkMain.java:178) >>>>> at mgm.tp.bigdata.ma_flink.FlinkMain.main(FlinkMain.java:47) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>> at >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>> at >>>>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) >>>>> at >>>>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353) >>>>> at org.apache.flink.client.program.Client.run(Client.java:315) >>>>> at >>>>> org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) >>>>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) >>>>> at >>>>> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:880) >>>>> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:922) >>>>> org.apache.flink.client.program.ProgramInvocationException: The >>>>> program execution failed: Failed to submit job >>>>> 934743a5c49c6d5e31c9e8201452e36d (KMeans Flink) >>>>> at org.apache.flink.client.program.Client.run(Client.java:412) >>>>> at org.apache.flink.client.program.Client.run(Client.java:355) >>>>> at org.apache.flink.client.program.Client.run(Client.java:348) >>>>> at >>>>> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) >>>>> at mgm.tp.bigdata.ma_flink.FlinkMain.main(FlinkMain.java:70) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>> at >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>> at >>>>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) >>>>> at >>>>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353) >>>>> at org.apache.flink.client.program.Client.run(Client.java:315) >>>>> at >>>>> org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) >>>>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) >>>>> at >>>>> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:880) >>>>> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:922) >>>>> Caused by: org.apache.flink.runtime.client.JobExecutionException: >>>>> Failed to submit job 934743a5c49c6d5e31c9e8201452e36d (KMeans Flink) >>>>> at org.apache.flink.runtime.jobmanager.JobManager.org >>>>> $apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:595) >>>>> at >>>>> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:192) >>>>> at >>>>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) >>>>> at >>>>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) >>>>> at >>>>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) >>>>> at >>>>> org.apache.flink.yarn.ApplicationMasterActor$$anonfun$receiveYarnMessages$1.applyOrElse(ApplicationMasterActor.scala:99) >>>>> at scala.PartialFunction$OrElse.apply(PartialFunction.scala:162) >>>>> at >>>>> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:36) >>>>> at >>>>> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:29) >>>>> at >>>>> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) >>>>> at >>>>> org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:29) >>>>> at akka.actor.Actor$class.aroundReceive(Actor.scala:465) >>>>> at >>>>> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:94) >>>>> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) >>>>> at akka.actor.ActorCell.invoke(ActorCell.scala:487) >>>>> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254) >>>>> at akka.dispatch.Mailbox.run(Mailbox.scala:221) >>>>> at akka.dispatch.Mailbox.exec(Mailbox.scala:231) >>>>> at >>>>> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) >>>>> at >>>>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >>>>> at >>>>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >>>>> at >>>>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >>>>> Caused by: org.apache.flink.runtime.JobException: Creating the input >>>>> splits caused an error: File /user/cloudera/outputs/seed-1 does not exist >>>>> or the user running Flink ('yarn') has insufficient permissions to access >>>>> it. >>>>> at >>>>> org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:162) >>>>> at >>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph.attachJobGraph(ExecutionGraph.java:471) >>>>> at org.apache.flink.runtime.jobmanager.JobManager.org >>>>> $apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:535) >>>>> ... 21 more >>>>> Caused by: java.io.FileNotFoundException: File >>>>> /user/cloudera/outputs/seed-1 does not exist or the user running Flink >>>>> ('yarn') has insufficient permissions to access it. >>>>> at >>>>> org.apache.flink.core.fs.local.LocalFileSystem.getFileStatus(LocalFileSystem.java:106) >>>>> at >>>>> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:390) >>>>> at >>>>> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:51) >>>>> at >>>>> org.apache.flink.runtime.executiongraph.ExecutionJobVertex.<init>(ExecutionJobVertex.java:146) >>>>> ... 23 more >>>>> >>>>> >>>>> how i must set the files in the hdfs? >>>>> quickstart.cloudera:50075/home/cloudera/output? >>>>> >>>>> 2015-06-04 16:51 GMT+02:00 Robert Metzger <rmetz...@apache.org>: >>>>> >>>>>> Once you've started the YARN session, you can submit a Flink job with >>>>>> "./bin/flink run <pathToYourJar>". >>>>>> >>>>>> The jar file of your job doesn't need to be in HDFS. It has to be in >>>>>> the local file system and flink will send it to all machines. >>>>>> >>>>>> On Thu, Jun 4, 2015 at 4:48 PM, Pa Rö <paul.roewer1...@googlemail.com >>>>>> > wrote: >>>>>> >>>>>>> okay, now it run on my hadoop. >>>>>>> how i can start my flink job? and where must the jar file save, at >>>>>>> hdfs or as local file? >>>>>>> >>>>>>> 2015-06-04 16:31 GMT+02:00 Robert Metzger <rmetz...@apache.org>: >>>>>>> >>>>>>>> Yes, you have to run these commands in the command line of the >>>>>>>> Cloudera VM. >>>>>>>> >>>>>>>> On Thu, Jun 4, 2015 at 4:28 PM, Pa Rö < >>>>>>>> paul.roewer1...@googlemail.com> wrote: >>>>>>>> >>>>>>>>> you mean run this command on terminal/shell and not define a hue >>>>>>>>> job? >>>>>>>>> >>>>>>>>> 2015-06-04 16:25 GMT+02:00 Robert Metzger <rmetz...@apache.org>: >>>>>>>>> >>>>>>>>>> It should be certainly possible to run Flink on a cloudera live VM >>>>>>>>>> >>>>>>>>>> I think these are the commands you need to execute: >>>>>>>>>> >>>>>>>>>> wget >>>>>>>>>> http://stratosphere-bin.s3-website-us-east-1.amazonaws.com/flink-0.9-SNAPSHOT-bin-hadoop2.tgz >>>>>>>>>> tar xvzf flink-0.9-SNAPSHOT-bin-hadoop2.tgz >>>>>>>>>> cd flink-0.9-SNAPSHOT/ >>>>>>>>>> *export HADOOP_CONF_DIR=/usr/lib/hadoop/etc/hadoop/* >>>>>>>>>> ./bin/yarn-session.sh -n 1 -jm 1024 -tm 1024 >>>>>>>>>> >>>>>>>>>> If that is not working for you, please post the exact error >>>>>>>>>> message you are getting and I can help you to get it to run. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jun 4, 2015 at 4:18 PM, Pa Rö < >>>>>>>>>> paul.roewer1...@googlemail.com> wrote: >>>>>>>>>> >>>>>>>>>>> hi robert, >>>>>>>>>>> >>>>>>>>>>> i think the problem is the hue api, >>>>>>>>>>> i had the same problem with spark submit script, >>>>>>>>>>> but on the new hue release, they have a spark submit api. >>>>>>>>>>> >>>>>>>>>>> i asked the group for the same problem with spark, no reply. >>>>>>>>>>> >>>>>>>>>>> i want test my app on local cluster, before i run it on the big >>>>>>>>>>> cluster, >>>>>>>>>>> for that i use cloudera live. maybe it give an other way to test >>>>>>>>>>> flink on a local cluster vm? >>>>>>>>>>> >>>>>>>>>>> 2015-06-04 16:12 GMT+02:00 Robert Metzger <rmetz...@apache.org>: >>>>>>>>>>> >>>>>>>>>>>> Hi Paul, >>>>>>>>>>>> >>>>>>>>>>>> why did running Flink from the regular scripts not work for you? >>>>>>>>>>>> >>>>>>>>>>>> I'm not an expert on Hue, I would recommend asking in the Hue >>>>>>>>>>>> user forum / mailing list: >>>>>>>>>>>> https://groups.google.com/a/cloudera.org/forum/#!forum/hue-user >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 4, 2015 at 4:09 PM, Pa Rö < >>>>>>>>>>>> paul.roewer1...@googlemail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> thanks, >>>>>>>>>>>>> now i want run my app on cloudera live vm single node, >>>>>>>>>>>>> how i can define my flink job with hue? >>>>>>>>>>>>> i try to run the flink script in the hdfs, it's not work. >>>>>>>>>>>>> >>>>>>>>>>>>> best regards, >>>>>>>>>>>>> paul >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-06-02 14:50 GMT+02:00 Robert Metzger <rmetz...@apache.org >>>>>>>>>>>>> >: >>>>>>>>>>>>> >>>>>>>>>>>>>> I would recommend using HDFS. >>>>>>>>>>>>>> For that, you need to specify the paths like this: >>>>>>>>>>>>>> hdfs:///path/to/data. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jun 2, 2015 at 2:48 PM, Pa Rö < >>>>>>>>>>>>>> paul.roewer1...@googlemail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> nice, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> which file system i must use for the cluster? java.io or >>>>>>>>>>>>>>> hadoop.fs or flink? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2015-06-02 14:29 GMT+02:00 Robert Metzger < >>>>>>>>>>>>>>> rmetz...@apache.org>: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> you can start Flink on YARN on the Cloudera distribution. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> See here for more: >>>>>>>>>>>>>>>> http://ci.apache.org/projects/flink/flink-docs-master/setup/yarn_setup.html >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> These are the commands you need to execute >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> wget >>>>>>>>>>>>>>>> http://stratosphere-bin.s3-website-us-east-1.amazonaws.com/flink-0.9-SNAPSHOT-bin-hadoop2.tgz >>>>>>>>>>>>>>>> tar xvzf flink-0.9-SNAPSHOT-bin-hadoop2.tgzcd >>>>>>>>>>>>>>>> flink-0.9-SNAPSHOT/ >>>>>>>>>>>>>>>> ./bin/yarn-session.sh -n 4 -jm 1024 -tm 4096 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Jun 2, 2015 at 2:03 PM, Pa Rö < >>>>>>>>>>>>>>>> paul.roewer1...@googlemail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> hi community, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> i want test my flink k-means on a hadoop cluster. i use >>>>>>>>>>>>>>>>> the cloudera live distribution. how i can run flink on this >>>>>>>>>>>>>>>>> cluster? maybe >>>>>>>>>>>>>>>>> only the java dependencies are engouth? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> best regards, >>>>>>>>>>>>>>>>> paul >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
