Hi,
I am running the RollingCount Benchmark from this set of benchmarks
<https://github.com/intel-hadoop/storm-benchmark>. Here is the relevant
piece of code:
spout = new FileReadSpout(BenchmarkUtils.ifAckEnabled(config));
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout(SPOUT_ID, spout, spoutNum);
builder.setBolt(SPLIT_ID, new WordCount.SplitSentence(), spBoltNum)
.localOrShuffleGrouping(SPOUT_ID);
builder.setBolt(COUNTER_ID, new RollingCountBolt(windowLength,
emitFreq), rcBoltNum)
.fieldsGrouping(SPLIT_ID, new Fields(WordCount.SplitSentence.FIELDS));
The FileReadSpout simply reads text from a file.
I have a three node setup with a total of 96 cores with spBoltNum = 6
and rcBoltNum
= 6. After a run, I see that there is a significant imbalance in the
capacity metric reported for each executor of the split bolt. Even though
each node has 2 executors for split bolt. I see the following numbers for
capacity of split bolt executors on each node:
Node 1 ~ 0.95
Node 2 ~ 0.7
Node 3 ~ 0.25
I do not understand this imbalance in utilization as the grouping for split
bolt is localOrShuffleGrouping, I was expecting the capacity reported for
each executor to be more or less equal. What am I missing here?
Here is the link to Stack Overflow question
<http://stackoverflow.com/questions/35556817/utilization-imbalance-in-storm-bolt-executors>
that I have posted.
Thanks.
Regards,
Bilal
