[OMPI users] Open MPI 4.0.3 outside as well as inside a SimpleFOAM container: step creation temporarily disabled, retrying Requested nodes are busy

Singularity 3.5.3 on RHEL 7 cluster w/ OpenMPI 4.0.3 lives inside a SimpleFOAM version 10 container. I've confirmed the OpenMPI versions are the same. Perhaps this is a question for Singularity users as well but how can I troubleshoot why mpirun just returns step creation temporarily disabled, retr

Re: [OMPI users] Open MPI 4.0.3 outside as well as inside a SimpleFOAM container: step creation temporarily disabled, retrying Requested nodes are busy

Rob, Do you invoke mpirun from **inside** the container? IIRC, mpirun is generally invoked from **outside** the container, could you try this if not already the case? The error message is from SLURM, so this is really a SLURM vs singularity issue. What if you srun -N 2 -n 2 hostname in