How many spouts do you have? On Mon, Feb 22, 2016 at 10:09 AM, Muhammad Bilal <[email protected]> wrote:
> Hi, > > I am running the RollingCount Benchmark from this set of benchmarks > <https://github.com/intel-hadoop/storm-benchmark>. Here is the relevant > piece of code: > > spout = new FileReadSpout(BenchmarkUtils.ifAckEnabled(config)); > > TopologyBuilder builder = new TopologyBuilder(); > > builder.setSpout(SPOUT_ID, spout, spoutNum); > builder.setBolt(SPLIT_ID, new WordCount.SplitSentence(), spBoltNum) > .localOrShuffleGrouping(SPOUT_ID); > builder.setBolt(COUNTER_ID, new RollingCountBolt(windowLength, emitFreq), > rcBoltNum) > .fieldsGrouping(SPLIT_ID, new Fields(WordCount.SplitSentence.FIELDS)); > > The FileReadSpout simply reads text from a file. > > I have a three node setup with a total of 96 cores with spBoltNum = 6 and > rcBoltNum > = 6. After a run, I see that there is a significant imbalance in the > capacity metric reported for each executor of the split bolt. Even though > each node has 2 executors for split bolt. I see the following numbers for > capacity of split bolt executors on each node: > > Node 1 ~ 0.95 > > Node 2 ~ 0.7 > > Node 3 ~ 0.25 > > I do not understand this imbalance in utilization as the grouping for > split bolt is localOrShuffleGrouping, I was expecting the capacity reported > for each executor to be more or less equal. What am I missing here? > > Here is the link to Stack Overflow question > <http://stackoverflow.com/questions/35556817/utilization-imbalance-in-storm-bolt-executors> > that I have posted. > > Thanks. > > Regards, > > Bilal >
