On Feb 9, 2006, at 3:03 PM, Jean-Christophe Hugly wrote:
On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote:
[...]
From an adoption perspective, though, the ability to shine in
micro-benchmarks is important, even if it means using an ad-hoc
tuning.
There is some justification for it after all. There are small
clusters
out there (many more than big ones, in fact) so taking maximum
advantage
of a small scale is relevant.
I'm obliged to point out that you jumped to a conclusion -- possibly
true
in some cases, but not always.
You assumed that a performance increase for a two-node micro-benchmark
would result in an application performance increase for a small
cluster.
Using RDMA for short messages is the default on small clusters
*because*
of the two-node micro-benchmark, not because the cluster is small.
No, I assumed it based on comparisions between doing and not doing
small
msg rdma at various scales, from a paper Galen pointed out to me.
http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf
Hmm, this is not what I would conclude from my results, in fact if you
look at the NPB results in my paper you will see that Open MPI
outperforms in the CG and FT benchmarks at both 32 and 64 nodes without
SRQ. The crossover point you are referring to must be the pairwise
ping-pong benchmark. So I would have to conclude that it is totally
application dependent.
- Galen
Benchmarks are what they are. In the above paper, the tests place the
cross-over at around 64 nodes and that confirms a number of anecdotal
reports I got. It may well be that in some situations, small-msg rdma
is
better only for 2 nodes, but that's note such a likely scenario;
reality
is sometimes linear (at least at our scale :-) ) after all.
The scale threshold could be tunable, couldnt it ?
--
Jean-Christophe Hugly <j...@pantasys.com>
PANTA
