Levi, as a workaround, have you tried using mpirun instead of direct launch (e.g. srun) ?
Note you are using pmix 1.2.5, so you likely want to srun --mpi=pmix_v1 Also, as reported by the logs 1. [nodeA:12838] OPAL ERROR: Error in file pmix3x_client.c at line 112 there is something fishy here, since there is no pmix3x_client.c in Open MPI 2.x (there is pmix1_client.c though) Cheers, Gilles On Wed, Apr 22, 2020 at 7:53 AM Levi D Davis via users < users@lists.open-mpi.org> wrote: > Hello, > > I have been having a heck of a time getting an old version of OMPI (v2.1) > with slurm (v19). I need it to work with an application that I don't have > control of recompiling with newer OMPI versions. > > I think it is an OMPI or PMIX compiling issue because the newer versions > (OMPI v3 and v4, PMIX v2 and v3) work with slurm as expected form the docs. > I am not sure if it is because of a parameter I need to set for an older > OMPI to make it working, since it is hard to find details about old > versions and issues. > > Here are some commands: > > srun --mpi=list > srun: MPI types are... > srun: pmix_v3 > srun: none > srun: openmpi > srun: pmi2 > srun: pmix > srun: pmix_v1 > srun: pmix_v2 > > srun --mpi=pmix_v2 -N 2 -n 2 mpi_program > Hello World from Node headnode, job 0 > Hello World from Node nodeA, job 1 > > > https://pastebin.com/5P8eJ7hb > <https://pastebin.com/5P8eJ7hb> > srun --mpi=pmix_v1 -N 2 -n 2 mpi_program [headnode:17246] PMIX ERROR: > UNPACK-PA - Pastebin.com <https://pastebin.com/5P8eJ7hb> > pastebin.com > > I have OMPI and pmix as modules which I load at runtime for the different > slurm runs (different pmix). > > Configure commands for OMPI v2 > > ./configure --prefix=/data_storage/cluster_software/mpis/openmpi-2.1.3 > --with-libevent=external --with-hwloc=external --with-ucx=/usr > --with-pmix=/data_storage/cluster_software/mpis/pmix-1.2.5 --with-slurm > --disable-pmix-dstore > > Configure command for PMIX v1 > ./configure --prefix=/data_storage/cluster_software/mpis/pmix-1.2.5 > > > > Anyone have an idea? > >
