Re: Flink Parallelism for various type of transformation

2020-07-06 Thread Arvid Heise
Hi Prasanna, overcommitting cores was actually a recommended technique a while ago to counter-balance I/O. So it's not bad per se. However, with slot sharing each core is already doing the work for source, transform, sink, so it's not necessary. So I'd go with slots = cores and I rather strongly

Flink Parallelism for various type of transformation

2020-07-06 Thread Prasanna kumar
Hi , I used t2.medium machines for the task manager nodes. It has 2 CPU and 4GB memory. But the task manager screen shows that there are 4 slots. Generally we should match the number of slots to the number of cores. [image: image.png] Our pipeline is Source -> Simple Transform -> Sink. What h