Re: [OMPI users] openMPI programs not using IB network

2015-10-16 Thread Gilles Gouaillardet
David, ib0 means IP over IB this is *not* what you want to use since it is way slower than native infiniband. if you mpirun --mca self,sm,openib ... on more than one node, the only btl usable for inter node communication is openib, so if communication happen, that means opening is used. in order

[OMPI users] openMPI programs not using IB network

2015-10-16 Thread David Arnold
Hi, We appear to have a correctly setup Mellanox IB network (ibdiagnet, ibstat, iblinkinfo, ibqueryerrors(*)). It's operating at Rate 40 FDR10. But openMPI programs (test and user) that are specifying the 'openib,self,sm' paramenters do not seem to be using the IB network according to network- m