Hi Arnaud, as long as you don't have HA activated for your batch jobs, HA shouldn't have an influence on the batch execution. If it interferes, then you should see additional task manager connected to the streaming cluster when you execute the batch job. Could you check that? Furthermore, could you check that actually a second yarn application is started when you run the batch jobs?
Cheers, Till On Thu, Dec 3, 2015 at 9:57 AM, LINZ, Arnaud <al...@bouyguestelecom.fr> wrote: > Hello, > > > > I have both streaming applications & batch applications. Since the memory > needs are not the same, I was using a long-living container for my > streaming apps and new short-lived containers for hosting each batch > execution. > > > > For that, I submit streaming jobs with "*flink run*" and batch jobs with > "*flink run -m yarn-cluster*" > > > > This was working fine until I turned zookeeper HA mode on for my streaming > applications. > > Even if I don't set it up in the yaml flink configuration file, but with > -D options on the yarn_session.sh command line, now my batch jobs try to > run in the streaming container, and fails because of the lack of ressources. > > > > My HA options are : > > -Dyarn.application-attempts=10 -Drecovery.mode=zookeeper > -Drecovery.zookeeper.quorum=h1r1en01:2181 > -Drecovery.zookeeper.path.root=/flink -Dstate.backend=filesystem > -Dstate.backend.fs.checkpointdir=hdfs:///tmp/flink/checkpoints > -Drecovery.zookeeper.storageDir=hdfs:///tmp/flink/recovery/ > > > > Am I missing something ? > > > > Best regards, > > Aranud > > ------------------------------ > > L'intégrité de ce message n'étant pas assurée sur internet, la société > expéditrice ne peut être tenue responsable de son contenu ni de ses pièces > jointes. Toute utilisation ou diffusion non autorisée est interdite. Si > vous n'êtes pas destinataire de ce message, merci de le détruire et > d'avertir l'expéditeur. > > The integrity of this message cannot be guaranteed on the Internet. The > company that sent this message cannot therefore be held liable for its > content nor attachments. Any unauthorized use or dissemination is > prohibited. If you are not the intended recipient of this message, then > please delete it and notify the sender. >
