Re: About nodes number on Flink

2017-06-26 Thread Timo Walther
If you really what to run one operation per node. You start 1 TaskManager with 1 slot on every node. For each operation you set a new chain and a new slot sharing group. Timo Am 23.06.17 um 15:03 schrieb AndreaKinn: Hi Timo, thanks for your answer. I think my elaboration are not too much heav

Re: About nodes number on Flink

2017-06-23 Thread AndreaKinn
Hi Timo, thanks for your answer. I think my elaboration are not too much heavy so I imagine I will have no advantages to "parallelize" streams. In my mind I have this pipeline: An

Re: About nodes number on Flink

2017-06-23 Thread Timo Walther
Hi Andrea, the number of nodes usually depends on the work that you do within your Functions. E.g. if you have a computation intensive machine learning library in a MapFunction and takes 10 seconds per element, it might make sense to paralellize this in order to increase your throughput. Or