Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Till Rohrmann
For standalone mode it works for the legacy and the new mode. Only when using Yarn or Mesos, it can happen that you temporarily allocate too many container. The unused container will, however, be freed if they idle too long. Cheers, Till On Fri, Jun 1, 2018, 21:52 Abdul Qadeer wrote: > Thank yo

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Abdul Qadeer
Thank you for creating this Till, so as I understand this won't be supported (for standalone cluster mode) even if legacy mode is enabled? On Fri, Jun 1, 2018 at 6:21 AM, Till Rohrmann wrote: > FYI: https://issues.apache.org/jira/browse/FLINK-9455 tracks the problem > with the multi slot support

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Till Rohrmann
FYI: https://issues.apache.org/jira/browse/FLINK-9455 tracks the problem with the multi slot support for the SlotManager. Cheers, Till On Fri, Jun 1, 2018 at 10:10 AM, Abdul Qadeer wrote: > Hi Kien! > > Thank you for sharing your experience. Do you have Flink deployed in > Standalone Cluster mo

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Abdul Qadeer
Hi Kien! Thank you for sharing your experience. Do you have Flink deployed in Standalone Cluster mode or K8S/Docker mode? On Fri, 1 Jun 2018 at 00:55, Fabian Hueske wrote: > Hi, > > The release notes state that "multiple slots are not *fully* supported". > > In Flink 1.5.0, the configured numbe

Re: Multiple Task Slots support in Flink 1.5

2018-06-01 Thread Fabian Hueske
Hi, The release notes state that "multiple slots are not *fully* supported". In Flink 1.5.0, the configured number of slots is ignored when requesting containers for TaskManagers from a resource manager, i.e., Flink assumes TMs with 1 slot. Hence, Flink request too many containers and starts too

Re: Multiple Task Slots support in Flink 1.5

2018-05-31 Thread Kien Truong
Hi, We're using multiple slots per TaskManager with legacy mode, and everything works fine. For the new default mode, it also seems to works for us, so I'm not sure what is not supported. May be someone from Flink team could clarify. Best regards, Kien On 5/31/2018 4:26 AM, Abdul Qadeer