[OMPI users] MPI_Comm_Spawn failure: All nodes already filled

2019-08-06 Thread Mccall, Kurt E. (MSFC-EV41) via users
Hi, MPI_Comm_spawn() is failing with the error message "All nodes which are allocated for this job are already filled". I compiled OpenMpi 4.0.1 with the Portland Group C++ compiler, v. 19.5.0, both with and without Torque/Maui support. I thought that not using Torque/Maui support would gi

[OMPI users] silent failure for large allgather

2019-08-06 Thread Emmanuel Thomé via users
Hi, In the attached program, the MPI_Allgather() call fails to communicate all data (the amount it communicates wraps around at 4G...). I'm running on an omnipath cluster (2018 hardware), openmpi 3.1.3 or 4.0.1 (tested both). With the OFI mtl, the failure is silent, with no error message reporte

Re: [OMPI users] MPI_Comm_Spawn failure: All nodes already filled

2019-08-06 Thread Ralph Castain via users
I'm afraid I cannot replicate this problem on OMPI master, so it could be something different about OMPI 4.0.1 or your environment. Can you download and test one of the nightly tarballs from the "master" branch and see if it works for you? https://www.open-mpi.org/nightly/master/ Ralph On Au