Thank you And i need this because i am developing a custom scheduler . And for this i need to have topologies performing well with two worker processes instead of one. And its becoming tough to saturate a single worker
On Saturday, 13 February 2016, Matthias J. Sax <[email protected]> wrote: > If your workload does not saturate a single machine, it will of course > be more efficient to run within a single worker as you avoid intra-JVM > communication. > > As long as CPU, memory, or network of a single machine is not utilized > completely, you will not benefit from multiple workers from a > performance point of view. > > Btw: I am a not sure what you mean by "I have to create a scenario..."? > > On 02/12/2016 11:42 PM, Rudraneel chakraborty wrote: > > Thanks. Actually i have to create a scenario where 2 worker performs > > better than one worker. But in reality , topology with a single worker > > performs considerably better. > > > > I sending csv lines to kafka (5 partitions ) and reading them from a > > topology with kafka spout (parallelism hint 5) > > > > Any thoughts? > > > > On Thursday, 11 February 2016, Nathan Leung <[email protected] > <javascript:;> > > <mailto:[email protected] <javascript:;>>> wrote: > > > > Any situation where you require more CPU than 1 server can provide > > for you - there are tuning parameters (e.g. localOrShuffleGrouping) > > that you can use to reduce the amount of data sent over the network > too. > > > > Any situation where you need to have tolerance in case of machine > > failure. > > > > > > > > On Thu, Feb 11, 2016 at 2:58 PM, Rudraneel chakraborty > > <[email protected] <javascript:;> > > <javascript:_e(%7B%7D,'cvml','[email protected] > <javascript:;>');>> > > wrote: > > > > Topology param numWorkers i meant > > > > > > On Thursday, 11 February 2016, Rudraneel chakraborty > > <[email protected] <javascript:;> > > <javascript:_e(%7B%7D,'cvml','[email protected] > <javascript:;>');>> > > wrote: > > > > More specifically , i have seen a topology performs better > > if it is assigned a single worker compared to more than one > > worker. > > > > I want a situation where a topology performs better with > > more than one worker. > > > > And it doesnt matter if both workers are on same supervisor > > or different supervisor > > > > On Thursday, 11 February 2016, Matthias J. Sax > > <[email protected] <javascript:;>> wrote: > > > > I am not sure what you mean: > > > > - number of worker slots per supervisor > > or > > - topology parameter "number of workers" > > > > Can you clarify? > > > > -Matthias > > > > On 02/11/2016 05:14 AM, anshu shukla wrote: > > > Not like that.. But i have used workers equal to > > number of cores. Each > > > vm with 8 corea. > > > > > > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" > > > <[email protected] <javascript:;> > > > <mailto:[email protected] <javascript:;>>> > wrote: > > > > > > more than one worker on same node ? Did u use > > custom schedule r? > > > because by default, the workers would be spread > > throughout the cluster > > > > > > > > > > > -- > > Rudraneel Chakraborty > > Carleton University Real Time and Distributed Systems > Reserach > > > > > > > > -- > > Rudraneel Chakraborty > > Carleton University Real Time and Distributed Systems Reserach > > > > > > > > > > -- > > Rudraneel Chakraborty > > Carleton University Real Time and Distributed Systems Reserach > > > > -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach
