Re: Source & job parallelism

2015-08-25 Thread Matthias J. Sax
Hi Arnaud, did you try: > Env.setSource(mySource).setParrellelism(1).map(mymapper).setParallelism(10) If this does not work, it might be that Flink chains the mapper to the source which implies to use the same parallelism (and the producer dictates this dop value). Using a rebalance() in betwee

Source & job parallelism

2015-08-25 Thread LINZ, Arnaud
Hi, I have a streaming source that extends RichParallelSourceFunction, but for some reason I don’t want parallelism at the source level, so I use : Env.setSource(mySource).setParrellelism(1).map(mymapper) I do want parallelism at the mapper level, because it’s a long task, and I would like the