Re: Using numberOfTaskSlots to control parallelism

2016-02-20 Thread Zach Cox
Thanks for the input Aljoscha and Ufuk! I will try out the #2 approach and report back. Thanks, Zach On Sat, Feb 20, 2016 at 7:26 AM Ufuk Celebi wrote: > On Sat, Feb 20, 2016 at 10:12 AM, Aljoscha Krettek > wrote: > > IMHO the only change for 2) is that you possibly get better machine > utili

Re: Using numberOfTaskSlots to control parallelism

2016-02-20 Thread Ufuk Celebi
On Sat, Feb 20, 2016 at 10:12 AM, Aljoscha Krettek wrote: > IMHO the only change for 2) is that you possibly get better machine > utilization because it will use more parallel threads. So I think it’s a > valid approach. > > @Ufuk, could there be problems with the number of network buffers? I t

Re: Using numberOfTaskSlots to control parallelism

2016-02-20 Thread Aljoscha Krettek
IMHO the only change for 2) is that you possibly get better machine utilization because it will use more parallel threads. So I think it’s a valid approach. @Ufuk, could there be problems with the number of network buffers? I think not, because the connections are multiplexed in one channel, is

Using numberOfTaskSlots to control parallelism

2016-02-19 Thread Zach Cox
What would the differences be between these scenarios? 1) one task manager with numberOfTaskSlots=1 and one job with parallelism=1 2) one task manager with numberOfTaskSlots=10 and one job with parallelism=10 In both cases all of the job's tasks get executed within the one task manager's jvm. Ar