Hi Xintong, At the moment I'm using the 1.9.2 with this command: yarn-session.sh -d *-s 1* -jm 4096 -tm 4096 -qu "XXX" -nm "MyPipeline" So, after a lot of tests, I've noticed that if I increase the parallelism of my Custom Sink, each task is embedded into one TS and, the most important, each one into one TaskManager (Yarn container in fact). So, if I understand I have to keep this Flink release (1.9.2) ?
Thanks David Le mar. 25 févr. 2020 à 02:02, Xintong Song <tonysong...@gmail.com> a écrit : > Depending on your Flink version, the '-n' option might not take effect. It > is removed in the latest release, but before that there were a few versions > where this option is neither removed nor taking effect. > > Anyway, as long as you have multiple containers, I don't think there's a > way to make some of the tasks scheduled to the same JVM. Not that I'm aware > of. > > > Thank you~ > > Xintong Song > > > > On Mon, Feb 24, 2020 at 8:43 PM David Morin <morin.david....@gmail.com> > wrote: > >> Hi, >> >> Thanks Xintong. >> I've noticed than when I use yarn-session.sh with --slots (-s) parameter >> but without --container (-n) it creates one task/slot per taskmanager. >> Before with the both n and -s it was not the case. >> I prefer to use only small container with only one task to scale my >> pipeline and of course to prevent from thread-safe issue >> Do you think I cannot be confident on that behaviour ? >> >> Regards, >> David >> >> On 2020/02/22 17:11:25, David Morin <morin.david....@gmail.com> wrote: >> > Hi, >> > My app is based on a lib that is not thread safe (yet...). >> > In waiting of the patch has been pushed, how can I be sure that my Sink >> that uses this lib is in one JVM ? >> > Context: I use one Yarn session and send my Flink jobs to this session >> > >> > Regards, >> > David >> > >> >
