Re: [OMPI users] infiniband with MPI

2012-07-31 Thread Jeff Squyres
On Jul 31, 2012, at 12:14 AM, Joen Chen wrote: > After reading the FAQ about OFED, I knew that openMPI can collaborate with > RoCE. Correct -- Open MPI can use RoCE interfaces, if they are available. > Moreover, using the RoCE make some overhead because the underlying network > layers. In my i

[OMPI users] infiniband with MPI

2012-07-31 Thread Joen Chen
Hi every one! After reading the FAQ about OFED, I knew that openMPI can collaborate with RoCE. Moreover, using the RoCE make some overhead because the underlying network layers. In my infiniband bandwidth testing, I get the 5Gbps using IPoIB and 12Gbps using RDMA. The performance gap is huge for m