[OMPI users] knem/openmpi performance?

2013-07-12 Thread Mark Dixon
Hi, I'm taking a look at knem, to see if it improves the performance of any applications on our QDR InfiniBand cluster, so I'm eager to hear about other people's experiences. This doesn't appear to have been discussed on this list before. I appreciate that any affect that knem will have is e

Re: [OMPI users] knem/openmpi performance?

2013-07-12 Thread Jeff Squyres (jsquyres)
FWIW: a long time ago (read: many Open MPI / knem versions ago), I did a few benchmarks with knem vs. no knem Open MPI installations. IIRC, I used the typical suspects like NetPIPE, the NPBs, etc. There was a modest performance improvement (I don't remember the numbers offhand); it was a small

Re: [OMPI users] knem/openmpi performance?

2013-07-12 Thread Mark Dixon
On Fri, 12 Jul 2013, Jeff Squyres (jsquyres) wrote: ... In short: doing 1 memcopy consumes half the memory bandwidth of 2 mem copies. So when you have lots of MPI processes competing for memory bandwidth, it turns out that having each MPI process use half the bandwidth is a Really Good Idea.