[OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread William Hay
We have a couple of clusters with Qlogic Infinipath/Intel TrueScale networking. While testing a kernel upgrade we find that the Truescale drivers will no longer build against recent RHEL kernels. Intel tells us that the Omnipath drivers will work for True Scale adapters so we install those. Basi

Re: [OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread Gilles Gouaillardet
William, In order to force PSM (aka Infinipath) you can mpirun --mca pml cm --mca mtl psm ... (Replace with psm2 for PSM2 (aka Omnipath) You can also mpirun --mca pml_base_verbose 10 --mca mtl_base_verbose 10 ... in order to collect some logs. Bottom line, pml/cm should be selected (instead

Re: [OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread Cabral, Matias A
Hi William, Couple other questions: - Please share how you ompi configure line looks like. - Please clarify which is/are the compat libraries you refer to. There are some that are actually for the opposite case: Making TS app/libs run on Omnipath. - As Gilles mentions, moving to a newer m

Re: [OMPI users] Installation of openmpi-1.10.7 fails

2018-01-22 Thread Edgar Gabriel
after some further investigation, I am fairly confident that this is not an MPI I/O problem. The input file input_tmp.in is generated in this sequence of instructions (which is in Modules/open_close_input_file.f90) ---   IF ( TRIM(input_file_) /= ' ' ) THEn ! ! copy file to be open

Re: [OMPI users] Installation of openmpi-1.10.7 fails

2018-01-22 Thread Edgar Gabriel
well, my final comment on this topic, as somebody suggested earlier in this email chain, if you provide the input with the -i argument instead of piping from standard input, things seem to work as far as I can see (disclaimer: I do not know what the final outcome should be. I just see that the