[OMPI users] PMIx + OpenMPI
Anyone successfully using PMIx with OpenMPI and SLURM? I have,
1. Installed an “external” version (1.1.5) of PMIx.
2. Patched SLURM 15.08.13 with the SchedMD-provided PMIx patch (results in an
mpi_pmix plugin along the lines of mpi_pmi2).
3. Built OpenMPI 2.0.1 (tried 2.0.3 as well).
However, when attempting to launch MPI apps (LAMMPS in this case), I get
[c9a-s2.ufhpc:08914] PMIX ERROR: UNREACHABLE in file
src/client/pmix_client.c at line 199
This comes from,
if (PMIX_SUCCESS != (ret=usock_connect((struct sockaddr *)address,
&sd))) {
PMIX_ERROR_LOG(ret);
return ret;
}
I’ve googled and looked at the archives and don’t see any other references to
this error. Don’t really see much about using OpenMPI with pmix at all. I
assumed the “server” side was embedded in orted or some such but maybe not.
What am I missing? Is there some server that needs to be started separately as
with mpd?
Thanks,
Charlie Taylor
Research Computing
University of Florida
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] PMIx + OpenMPI
> On Aug 6, 2017, at 6:53 AM, Charles A Taylor wrote: > > > Anyone successfully using PMIx with OpenMPI and SLURM? I have, > > 1. Installed an “external” version (1.1.5) of PMIx. > 2. Patched SLURM 15.08.13 with the SchedMD-provided PMIx patch (results in an > mpi_pmix plugin along the lines of mpi_pmi2). > 3. Built OpenMPI 2.0.1 (tried 2.0.3 as well). > > However, when attempting to launch MPI apps (LAMMPS in this case), I get > >[c9a-s2.ufhpc:08914] PMIX ERROR: UNREACHABLE in file > src/client/pmix_client.c at line 199 > I should have mentioned that I’m launching with srun —mpi=pmix … If I launch with srun —mpi=pmi2 ... the app starts and runs without issue. ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] PMIx + OpenMPI
Charles, did you build Open MPI with the external PMIx ? iirc, Open MPI 2.0.x does not support cross version PMIx Cheers, Gilles On Sun, Aug 6, 2017 at 7:59 PM, Charles A Taylor wrote: > >> On Aug 6, 2017, at 6:53 AM, Charles A Taylor wrote: >> >> >> Anyone successfully using PMIx with OpenMPI and SLURM? I have, >> >> 1. Installed an “external” version (1.1.5) of PMIx. >> 2. Patched SLURM 15.08.13 with the SchedMD-provided PMIx patch (results in >> an mpi_pmix plugin along the lines of mpi_pmi2). >> 3. Built OpenMPI 2.0.1 (tried 2.0.3 as well). >> >> However, when attempting to launch MPI apps (LAMMPS in this case), I get >> >>[c9a-s2.ufhpc:08914] PMIX ERROR: UNREACHABLE in file >> src/client/pmix_client.c at line 199 >> > I should have mentioned that I’m launching with > >srun —mpi=pmix … > > If I launch with > > srun —mpi=pmi2 ... > > the app starts and runs without issue. > > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] PMIx + OpenMPI
HI Gilles, I tried both “—with-pmix=/opt/pmix” and “—with-pmix=internal” and got the same “UNREACHABLE” error both ways. I tried the “external” first since that is what SLURM was built against. I’m missing something simple/basic - just not sure what it is. Thanks, Charlie > On Aug 6, 2017, at 7:43 AM, Gilles Gouaillardet > wrote: > > Charles, > > did you build Open MPI with the external PMIx ? > iirc, Open MPI 2.0.x does not support cross version PMIx > > Cheers, > > Gilles > > On Sun, Aug 6, 2017 at 7:59 PM, Charles A Taylor wrote: >> >>> On Aug 6, 2017, at 6:53 AM, Charles A Taylor wrote: >>> >>> >>> Anyone successfully using PMIx with OpenMPI and SLURM? I have, >>> >>> 1. Installed an “external” version (1.1.5) of PMIx. >>> 2. Patched SLURM 15.08.13 with the SchedMD-provided PMIx patch (results in >>> an mpi_pmix plugin along the lines of mpi_pmi2). >>> 3. Built OpenMPI 2.0.1 (tried 2.0.3 as well). >>> >>> However, when attempting to launch MPI apps (LAMMPS in this case), I get >>> >>> [c9a-s2.ufhpc:08914] PMIX ERROR: UNREACHABLE in file >>> src/client/pmix_client.c at line 199 >>> >> I should have mentioned that I’m launching with >> >> srun —mpi=pmix … >> >> If I launch with >> >> srun —mpi=pmi2 ... >> >> the app starts and runs without issue. >> >> >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] PMIx + OpenMPI
This is just a guess but - is it possible after building slurm against PMIx, you should build openmpi against SLURM's PMIx instead of directly using external? I would assume slurm's PMI server now "knows" PMIx. On 06/08//2017 16:14, Charles A Taylor wrote: HI Gilles, I tried both “—with-pmix=/opt/pmix” and “—with-pmix=internal” and got the same “UNREACHABLE” error both ways. I tried the “external” first since that is what SLURM was built against. I’m missing something simple/basic - just not sure what it is. Thanks, Charlie On Aug 6, 2017, at 7:43 AM, Gilles Gouaillardet wrote: Charles, did you build Open MPI with the external PMIx ? iirc, Open MPI 2.0.x does not support cross version PMIx Cheers, Gilles On Sun, Aug 6, 2017 at 7:59 PM, Charles A Taylor wrote: On Aug 6, 2017, at 6:53 AM, Charles A Taylor wrote: Anyone successfully using PMIx with OpenMPI and SLURM? I have, 1. Installed an “external” version (1.1.5) of PMIx. 2. Patched SLURM 15.08.13 with the SchedMD-provided PMIx patch (results in an mpi_pmix plugin along the lines of mpi_pmi2). 3. Built OpenMPI 2.0.1 (tried 2.0.3 as well). However, when attempting to launch MPI apps (LAMMPS in this case), I get [c9a-s2.ufhpc:08914] PMIX ERROR: UNREACHABLE in file src/client/pmix_client.c at line 199 I should have mentioned that I’m launching with srun —mpi=pmix … If I launch with srun —mpi=pmi2 ... the app starts and runs without issue. ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users smime.p7s Description: S/MIME Cryptographic Signature ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
