Re: [OMPI users] OpenMPI 4.0.5 error with Omni-path

2021-01-26 Thread Patrick Begou via users
Hi Ralph and Michael, ofi is not available with my default build on this host (I just set --without-verbs) dahu112 : mpirun -np 4 --mca mtl ofi ./test_layout_array -- A requested component was not found, or was un

Re: [OMPI users] OpenMPI 4.0.5 error with Omni-path

2021-01-26 Thread Patrick Begou via users
Hi all, I ran many tests today. I saw that an older 4.0.2 version of OpenMPI packaged with Nix was running using openib. So I add the --with-verbs option to setup this module. That I can see now is that: mpirun -hostfile $OAR_NODEFILE *--mca mtl psm -mca btl_openib_allow_ib true* - the

Re: [OMPI users] OpenMPI 4.0.5 error with Omni-path

2021-01-26 Thread Heinz, Michael William via users
Patrick how are you using original PSM if you’re using Omni-Path hardware? The original PSM was written for QLogic DDR and QDR Infiniband adapters. As far as needing openib - the issue is that the PSM2 MTL doesn’t support a subset of MPI operations that we previously used the pt2pt BTL for. For

[OMPI users] OpenMPI version compatibility with libfabric, UCX, etc...

2021-01-26 Thread Craig via users
Is there a table somewhere that tells me what version of things like libfabric and UCX (and maybe compiler versions if there are known issues) are known to be good with which versions of OpenMPI?  I poked around in the README, FAQ, Download sections and for the life of me can't seem to find

Re: [OMPI users] OpenMPI 4.0.5 error with Omni-path

2021-01-26 Thread Patrick Begou via users
Hi Michael indeed I'm a little bit lost with all these parameters in OpenMPI, mainly because for years it works just fine out of the box in all my deployments on various architectures, interconnects and linux flavor. Some weeks ago I deploy OpenMPI4.0.5 in Centos8 with gcc10, slurm and UCX on an A