Hi Arnaud, I think my answer to Gwenhaël could also be helpful to you: are you using the one-yarn-cluster-per-job mode of Flink? I.e., you are starting your Flink job with (from the doc):
flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar If you are, then this is almost possible on the current version of Flink. What you have to do is copy the conf directory of Flink to a separate directory that is specific to your job. There you make your modifications to the log configuration etc. Then, when you start your job you do this instead: export FLINK_CONF_DIR=/path/to/my/conf flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar You can easily put this into your startup script, of course. I said almost possible because this requires a small fix in bin/flink. Around line 130 this line: FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf needs to be replaced by this line: if [ -z "$FLINK_CONF_DIR" ]; then FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf; fi (We will fix this in the upcoming version and the 0.9.1 bugfix release.) Does this help? Let us know if you are not using the one-yarn-cluster-per-job mode, then we'll have to try to find another solution. Forget the stuff about one-yarn-cluster-per-job but the other stuff should be right what you want. I hope it helps. Best, Aljoscha On Wed, 26 Aug 2015 at 15:19 LINZ, Arnaud <al...@bouyguestelecom.fr> wrote: > Ooops… Seems it was rather a write problem on the conf dir… > > Sorry, it works! > > > > BTW, it’s not really nice to have an application write the configuration > dir ; it’s often a root protected directory in usr/lib/flink. Is there a > parameter to put that file elsewhere ? > > > > > > *De :* Robert Metzger [mailto:rmetz...@apache.org] > *Envoyé :* mercredi 26 août 2015 14:42 > *À :* user@flink.apache.org > *Objet :* Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session > -nm), flink run without -m fails. > > > > Hi Arnaud, > > > > usually, you don't have to manually specify the JobManager address > manually with the -m argument, because it is reading it from the > conf/.yarn-session.properties file. > > > > Give me a few minutes to reproduce the issue. > > > > On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud <al...@bouyguestelecom.fr> > wrote: > > Hi, > Using last nightly build, it seems that if you call yarn-session.sh with > -nm option to give a nice application name, then you cannot submit a job > with flink run without specify the ever changing -m <jobManager> address > since it does not find it any longer. > > Regards, > > Arnaud > > > ------------------------------ > > > L'intégrité de ce message n'étant pas assurée sur internet, la société > expéditrice ne peut être tenue responsable de son contenu ni de ses pièces > jointes. Toute utilisation ou diffusion non autorisée est interdite. Si > vous n'êtes pas destinataire de ce message, merci de le détruire et > d'avertir l'expéditeur. > > The integrity of this message cannot be guaranteed on the Internet. The > company that sent this message cannot therefore be held liable for its > content nor attachments. Any unauthorized use or dissemination is > prohibited. If you are not the intended recipient of this message, then > please delete it and notify the sender. > > >
