I would run the v4.x series and install xpmem if you can (http://github.com/hjelmn/xpmem <http://github.com/hjelmn/xpmem>). You will need to build with —with-xpmem=/path/to/xpmem to use xpmem otherwise vader will default to using CMA. This will provide the best possible performance.
-Nathan > On Mar 4, 2021, at 5:55 AM, Raut, S Biplab via users > <users@lists.open-mpi.org> wrote: > > [AMD Official Use Only - Internal Distribution Only] > > It is a single node execution, so it should be using shared memory (vader). > > With Regards, > S. Biplab Raut > > From: Heinz, Michael William <michael.william.he...@cornelisnetworks.com > <mailto:michael.william.he...@cornelisnetworks.com>> > Sent: Thursday, March 4, 2021 5:17 PM > To: Open MPI Users <users@lists.open-mpi.org > <mailto:users@lists.open-mpi.org>> > Cc: Raut, S Biplab <biplab.r...@amd.com <mailto:biplab.r...@amd.com>> > Subject: Re: [OMPI users] Stable and performant openMPI version for > Ubuntu20.04 ? > > [CAUTION: External Email] > What interconnect are you using at run time? That is, are you using Ethernet > or InfiniBand or Omnipath? > > Sent from my iPad > > > On Mar 4, 2021, at 5:05 AM, Raut, S Biplab via users > <users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>> wrote: > > > [AMD Official Use Only - Internal Distribution Only] > > After downloading a particular openMPI version, let’s say v3.1.1 from > https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.1.tar.gz > <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownload.open-mpi.org%2Frelease%2Fopen-mpi%2Fv3.1%2Fopenmpi-3.1.1.tar.gz&data=04%7C01%7CBiplab.Raut%40amd.com%7C9e9baf59cf66417e2e1808d8df0344e3%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637504552448339561%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eeLrTeKE%2FwYobg0t0WN8TrGsbJ8xfjq6lJQgxptImUw%3D&reserved=0> > , I follow the below steps. > ./configure --prefix="$INSTALL_DIR" --enable-mpi-fortran --enable-mpi-cxx > --enable-shared=yes --enable-static=yes --enable-mpi1-compatibility > make -j > make install > export PATH=$INSTALL_DIR/bin:$PATH > export LD_LIBRARY_PATH=$INSTALL_DIR/lib:$LD_LIBRARY_PATH > Additionally, I also install libnuma-dev on the machine. > > For all the machines having Ubuntu 18.04 and 19.04, it works correctly and > results in expected performance/GFLOPS. > But, when OS is changed to Ubuntu 20.04, then I start getting the issues as > mentioned in my original/previous mail below. > > With Regards, > S. Biplab Raut > > From: users <users-boun...@lists.open-mpi.org > <mailto:users-boun...@lists.open-mpi.org>> On Behalf Of John Hearns via users > Sent: Thursday, March 4, 2021 1:53 PM > To: Open MPI Users <users@lists.open-mpi.org > <mailto:users@lists.open-mpi.org>> > Cc: John Hearns <hear...@gmail.com <mailto:hear...@gmail.com>> > Subject: Re: [OMPI users] Stable and performant openMPI version for > Ubuntu20.04 ? > > [CAUTION: External Email] > How are you installing the OpenMPI versions? Are you using packages which are > distributed by the OS? > > It might be worth looking at using Easybuid or Spack > https://docs.easybuild.io/en/latest/Introduction.html > <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.easybuild.io%2Fen%2Flatest%2FIntroduction.html&data=04%7C01%7CBiplab.Raut%40amd.com%7C9e9baf59cf66417e2e1808d8df0344e3%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637504552448339561%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qgr6YMUWZ184t3OE6UwRvE3lLdQLqAy26kr5ygoTctM%3D&reserved=0> > https://spack.readthedocs.io/en/latest/ > <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspack.readthedocs.io%2Fen%2Flatest%2F&data=04%7C01%7CBiplab.Raut%40amd.com%7C9e9baf59cf66417e2e1808d8df0344e3%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637504552448349558%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XauzRNeAfMI1mpKrbhL7UeWs4UrTMtZjfgmX29BAN2Y%3D&reserved=0> > > > On Thu, 4 Mar 2021 at 07:35, Raut, S Biplab via users > <users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>> wrote: > [AMD Official Use Only - Internal Distribution Only] > > Dear Experts, > Until recently, I was using openMPI3.1.1 to run > single node 128 ranks MPI application on Ubuntu18.04 and Ubuntu19.04. > But, now the OS on these machines are upgraded to Ubuntu20.04, and I have > been observing program hangs with openMPI3.1.1 version. > So, I tried with openMPI4.0.5 version – The program ran properly without any > issues but there is a performance regression in my application. > > Can I know the stable openMPI version recommended for Ubuntu20.04 that has no > known regression compared to v3.1.1. > > With Regards, > S. Biplab Raut
