On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote: > > [...] > > > > >From an adoption perspective, though, the ability to shine in > > micro-benchmarks is important, even if it means using an ad-hoc tuning. > > There is some justification for it after all. There are small clusters > > out there (many more than big ones, in fact) so taking maximum advantage > > of a small scale is relevant. > > I'm obliged to point out that you jumped to a conclusion -- possibly true > in some cases, but not always. > > You assumed that a performance increase for a two-node micro-benchmark > would result in an application performance increase for a small cluster. > Using RDMA for short messages is the default on small clusters *because* > of the two-node micro-benchmark, not because the cluster is small.
No, I assumed it based on comparisions between doing and not doing small msg rdma at various scales, from a paper Galen pointed out to me. http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf Benchmarks are what they are. In the above paper, the tests place the cross-over at around 64 nodes and that confirms a number of anecdotal reports I got. It may well be that in some situations, small-msg rdma is better only for 2 nodes, but that's note such a likely scenario; reality is sometimes linear (at least at our scale :-) ) after all. The scale threshold could be tunable, couldnt it ? -- Jean-Christophe Hugly <j...@pantasys.com> PANTA
