Hi Bert,
If you'd prefer to return to the land of convenience and don't need to mix
MPI
and OpenSHMEM, then you may want to try the path I outlined in the email
archived at the following link
https://www.mail-archive.com/users@lists.open-mpi.org/msg32274.html
Howard
Am Di., 13. Nov. 2018 um 23:10 Uhr schrieb Bert Wesarg via users <
users@lists.open-mpi.org>:
> Dear Takahiro,
> On Wed, Nov 14, 2018 at 5:38 AM Kawashima, Takahiro
> wrote:
> >
> > XPMEM moved to GitLab.
> >
> > https://gitlab.com/hjelmn/xpmem
>
> the first words from the README aren't very pleasant to read:
>
> This is an experimental version of XPMEM based on a version provided by
> Cray and uploaded to https://code.google.com/p/xpmem. This version
> supports
> any kernel 3.12 and newer. *Keep in mind there may be bugs and this version
> may cause kernel panics, code crashes, eat your cat, etc.*
>
> Installing this on my laptop where I just want developing with SHMEM
> it would be a pitty to lose work just because of that.
>
> Best,
> Bert
>
> >
> > Thanks,
> > Takahiro Kawashima,
> > Fujitsu
> >
> > > Hello Bert,
> > >
> > > What OS are you running on your notebook?
> > >
> > > If you are running Linux, and you have root access to your system,
> then
> > > you should be able to resolve the Open SHMEM support issue by
> installing
> > > the XPMEM device driver on your system, and rebuilding UCX so it picks
> > > up XPMEM support.
> > >
> > > The source code is on GitHub:
> > >
> > > https://github.com/hjelmn/xpmem
> > >
> > > Some instructions on how to build the xpmem device driver are at
> > >
> > > https://github.com/hjelmn/xpmem/wiki/Installing-XPMEM
> > >
> > > You will need to install the kernel source and symbols rpms on your
> > > system before building the xpmem device driver.
> > >
> > > Hope this helps,
> > >
> > > Howard
> > >
> > >
> > > Am Di., 13. Nov. 2018 um 15:00 Uhr schrieb Bert Wesarg via users <
> > > users@lists.open-mpi.org>:
> > >
> > > > Hi,
> > > >
> > > > On Mon, Nov 12, 2018 at 10:49 PM Pritchard Jr., Howard via announce
> > > > wrote:
> > > > >
> > > > > The Open MPI Team, representing a consortium of research,
> academic, and
> > > > > industry partners, is pleased to announce the release of Open MPI
> version
> > > > > 4.0.0.
> > > > >
> > > > > v4.0.0 is the start of a new release series for Open MPI.
> Starting with
> > > > > this release, the OpenIB BTL supports only iWarp and RoCE by
> default.
> > > > > Starting with this release, UCX is the preferred transport
> protocol
> > > > > for Infiniband interconnects. The embedded PMIx runtime has been
> updated
> > > > > to 3.0.2. The embedded Romio has been updated to 3.2.1. This
> > > > > release is ABI compatible with the 3.x release streams. There have
> been
> > > > numerous
> > > > > other bug fixes and performance improvements.
> > > > >
> > > > > Note that starting with Open MPI v4.0.0, prototypes for several
> > > > > MPI-1 symbols that were deleted in the MPI-3.0 specification
> > > > > (which was published in 2012) are no longer available by default in
> > > > > mpi.h. See the README for further details.
> > > > >
> > > > > Version 4.0.0 can be downloaded from the main Open MPI web site:
> > > > >
> > > > > https://www.open-mpi.org/software/ompi/v4.0/
> > > > >
> > > > >
> > > > > 4.0.0 -- September, 2018
> > > > >
> > > > >
> > > > > - OSHMEM updated to the OpenSHMEM 1.4 API.
> > > > > - Do not build OpenSHMEM layer when there are no SPMLs available.
> > > > > Currently, this means the OpenSHMEM layer will only build if
> > > > > a MXM or UCX library is found.
> > > >
> > > > so what is the most convenience way to get SHMEM working on a single
> > > > shared memory node (aka. notebook)? I just realized that I don't have
> > > > a SHMEM since Open MPI 3.0. But building with UCX does not help
> > > > either. I tried with UCX 1.4 but Open MPI SHMEM
> > > > still does not work:
> > > >
> > > > $ oshcc -o shmem_hello_world-4.0.0
> openmpi-4.0.0/examples/hello_oshmem_c.c
> > > > $ oshrun -np 2 ./shmem_hello_world-4.0.0
> > > > [1542109710.217344] [tudtug:27715:0] select.c:406 UCX ERROR
> > > > no remote registered memory access transport to tudtug:27716:
> > > > self/self - Destination is unreachable, tcp/enp0s31f6 - no put short,
> > > > tcp/wlp61s0 - no put short, mm/sysv - Destination is unreachable,
> > > > mm/posix - Destination is unreachable, cma/cma - no put short
> > > > [1542109710.217344] [tudtug:27716:0] select.c:406 UCX ERROR
> > > > no remote registered memory access transport to tudtug:27715:
> > > > self/self - Destination is unreachable, tcp/enp0s31f6 - no put short,
> > > > tcp/wlp61s0 - no put short, mm/sysv - Destination is unreachable,
> > > > mm/posix - Destination is unreachable, cma/cma - no put short
> > > > [tudtug:27715] ../../../../../oshmem/mca/spml/ucx/spml_ucx.c:266
> > > > Error: ucp_ep_create(proc=1/2) failed: Destination is unreachable
> > > > [tudtug:27715] ../
