Re: [OMPI users] Enforce TCP with mpirun

2017-08-17 Thread Maksym Planeta
Thanks, it worked out. On 08/16/2017 04:05 PM, Gilles Gouaillardet wrote: My bad, i forgot btl/tcp is using all the interfaces by default (eth0 *and* ib0) is eth0 available on all your nodes or just the node running mpirun ? you can try to use a subnet instead of an interface name mpirun --mc

Re: [OMPI users] Enforce TCP with mpirun

2017-08-16 Thread Gilles Gouaillardet
My bad, i forgot btl/tcp is using all the interfaces by default (eth0 *and* ib0) is eth0 available on all your nodes or just the node running mpirun ? you can try to use a subnet instead of an interface name mpirun --mca btl_tcp_if_include 172.24.44.0/23 ... if you are still facing some issues,

Re: [OMPI users] Enforce TCP with mpirun

2017-08-16 Thread Maksym Planeta
Dear Gilles, thank you for quick response. pml/cm doesn't work at all When I use "--mca pml ob1" I still see traffic in /usr/sbin/perfquery, but the program starts running a lot slower. E. g. ib.C.64 benchmarks runs 33 seconds in contrast to less than 1. I also see many if following warning

Re: [OMPI users] Enforce TCP with mpirun

2017-08-16 Thread Gilles Gouaillardet
Hi, you can try mpirun --mca pml ob1 --mca btl tcp,self ... pml/cm has a higher priority than pml/ob1, so if you have a mtl that fits your network (such as mtl/mxm), then pml/ob1 will be ignored, and the list of allowed/excluded btl become insignificant. Cheers, Gilles On Wed, Aug 16, 2017 at

[OMPI users] Enforce TCP with mpirun

2017-08-16 Thread Maksym Planeta
Hello, I work with an Infiniband cluster, but I want to force OpenMPI to use specific network interface. I tried to do this for example using mpirun as follows: mpirun --map-by node --mca btl self,tcp -np 16 bin/is.C.16 But counters returned by /usr/sbin/perfquery still keep showing that t