Hi Leon, Basically you are trading away utilizing all of your resources in the cluster to cut network IO. With putting more and more tasks into the same slot you are pushing a job from being potentially network bound to usually CPU bound, as the parallel instances of tasks in the same slot sharing group are scheduled on the same machine.
Have a look at the basic metrics of the working nodes of the cluster to determine when to stop sharing. Best, Marton On Sat, Jun 4, 2016 at 10:09 AM, <leon_mcl...@tutanota.com> wrote: > Hi, > > i am deploying a 5 stage pipeline, with varying DOPs for experimental > purposes. The benefits of slot sharing are clear. What is unclear to me is > when slot sharing would be undesirable. > > Since slot sharing enables bypassing of additional buffers (buffers > between slots) as well as memory based optimizations, it surely is always > beneficial. Furthermore, since all slots run within the same JVM, there > also appears to be no advantage of disabling slot sharing in regard to > simplifying JVM testing/debugging or for other isolation purposes. > > Could you perhaps elaborate on when slot sharing may be undesirable? > > Regards > Leon >
