Aws already has auto scale flink cluster it’s called Kinesis Data Analytics
just add your flink Jar to Kinesis Sql analytics that’s all , aws will auto
provision a flink cluster and do the admin part for you.
On Saturday, September 28, 2019, David Anderson wrote:
> I believe there can be advanta
I believe there can be advantages and disadvantages in both
directions. For example, fewer containers with multiple slots reduces
the effort the Flink Master has to do whenever global coordination is
required, i.e., during checkpointing. And the network stack in the
task managers is optimized to ta
Thanks Terry, the reason why I asked this is because somewhere I saw
running one slot per container is beneficial. I couldn’t find the where I
saw that.
Also I think running it with multiple slots will reduce IPC since some of
the data will be processed writhing the same JVM.
Thanks
On Wed, Sep 2
Hi, Navneeth,
I think both is ok.
IMO, run one container with number of slots same as virtual cores may be better
for slots can share the Flink Framework and thus reduce memory cost.
Best,
Terry Wang
> 在 2019年9月25日，下午3:26，Navneeth Krishnan 写道：
>
> Hi All,
>
> I’m currently running flink o
Hi All,
I’m currently running flink on amazon ecs and I have assigned task slots
based on vcpus per instance. Is it beneficial to run a separate container
with one slot each or one container with number of slots same as virtual
cores?
Thanks
