Are you able to give some detail on in which cases you might be better off setting higher (or lower) parallelism for an operator?
On Thu, Feb 21, 2019 at 9:54 PM Hung <unicorn.bana...@gmail.com> wrote: > / Each job has 3 asynch operators > with Executors with thread counts of 20,20,100/ > > Flink handles parallelisms for you. If you want a higher parallelism of a > operator, you can call setParallelism() > for example, > > flatMap(new Mapper1()).setParallelism(20) > flatMap(new Mapper2()).setParallelism(20) > flatMap(new Mapper3()).setParallelism(100) > > You can check the official document here > > https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/parallel.html#setting-the-parallelism > > /Currently we are using parallelism = 1/ > I guess you set the job level parallelism > > I would suggest you replace Executors with the use of Flink parallelisms. > It > would be more efficient so > you don't create the other thread pool although you already have one that > flink provides you(I maybe not right describing this concept) > > Cheers, > > Sendoh > > > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
