Re: [OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-28 Thread r...@open-mpi.org
The root cause is that the nodes are defined as “heterogeneous” because the difference in HCAs causes a difference in selection logic. For scalability purposes, we don’t circulate the choice of PML as that isn’t something mpirun can “discover” and communicate. One option we could pursue is to p

Re: [OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-28 Thread Orion Poplawski
On 02/27/2017 05:19 PM, Howard Pritchard wrote: > Hi Orion > > Does the problem occur if you only use font2 and 3? Do you have MXM installed > on the font1 node? No, running across font2/3 is fine. No idea what MXM is. > The 2.x series is using PMIX and it could be that is impacting the PML sa

Re: [OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-27 Thread Howard Pritchard
Hi Orion Does the problem occur if you only use font2 and 3? Do you have MXM installed on the font1 node? The 2.x series is using PMIX and it could be that is impacting the PML sanity check. Howard Orion Poplawski schrieb am Mo. 27. Feb. 2017 um 14:50: > We have a couple nodes with differen

[OMPI users] Issues with different IB adapters and openmpi 2.0.2

2017-02-27 Thread Orion Poplawski
We have a couple nodes with different IB adapters in them: font1/var/log/lspci:03:00.0 InfiniBand [0c06]: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] [15b3:6274] (rev 20) font2/var/log/lspci:03:00.0 InfiniBand [0c06]: QLogic Corp. IBA7220 InfiniBand HCA [1077:7220] (rev 02) font3/var/log