Interesting, we are using 3.0 that is in MOFED, and that is also what is on the MXM download site. Kinda confusing.
Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion bro...@umich.edu (734)936-1985 On Aug 28, 2014, at 2:12 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote: > btw, you may want to use latest mxm v3.1 which is part of hpcx package > http://www.mellanox.com/products/hpcx > > > > On Thu, Aug 28, 2014 at 4:10 AM, Brock Palen <bro...@umich.edu> wrote: > Brice, et al. > > Thanks a lot for this info. We are setting up new builds of OMPI 1.8.2 with > knem and mxm 3.0, > > If we have questions we will let you know. > > Brock Palen > www.umich.edu/~brockp > CAEN Advanced Computing > XSEDE Campus Champion > bro...@umich.edu > (734)936-1985 > > > > On Aug 27, 2014, at 12:44 PM, Brice Goglin <brice.gog...@inria.fr> wrote: > > > Hello Brock, > > > > Some people complained that giving world-wide access to a device file by > > default might be bad if we ever find a security leak in the kernel module. > > So I needed a better default. The rdma group is often used for OFED > > devices, and OFED and KNEM users are often the same, so it was a good > > compromise. > > > > There's no major issue with opening /dev/knem to everybody. A remote > > process memory is only accessible if an attacker finds the corresponding > > 64bit cookie. Only the memory buffer that was explicitly made readable > > and/or writable can be accessed read and/or write through this cookie. And > > recent KNEM releases also enforce by default that the attacker has the same > > uid as the target process. > > > > Brice > > > > > > > > > > Le 27/08/2014 16:25, Brock Palen a écrit : > >> Is there any major issues letting all users use it by setting /dev/knem to > >> 666 ? It appears knem by default wants to only allow users of the rdma > >> group (if defined) to access knem. > >> > >> We are a generic provider and want everyone to be able to use it, just > >> feels strange to restrict it, so I am trying to understand why that is the > >> default. > >> > >> Brock Palen > >> > >> www.umich.edu/~brockp > >> > >> CAEN Advanced Computing > >> XSEDE Campus Champion > >> > >> bro...@umich.edu > >> > >> (734)936-1985 > >> > >> > >> > >> On Aug 27, 2014, at 10:15 AM, Alina Sklarevich > >> <ali...@dev.mellanox.co.il> > >> wrote: > >> > >> > >>> Hi, > >>> > >>> KNEM can improve the performance significantly for intra-node > >>> communication and that's why MXM is using it. > >>> If you don't want to use it, you can suppress this warning by adding the > >>> following to your command line after mpirun: > >>> -x MXM_LOG_LEVEL=error > >>> > >>> Alina. > >>> > >>> > >>> On Wed, Aug 27, 2014 at 4:28 PM, Brock Palen > >>> <bro...@umich.edu> > >>> wrote: > >>> We updated our ofed and started to rebuild our MPI builds with mxm 3.0 . > >>> > >>> Now we get warnings bout knem > >>> > >>> [1409145437.578861] [flux-login1:31719:0] shm.c:65 MXM WARN > >>> Could not open the KNEM device file at /dev/knem : No such file or > >>> directory. Won't use knem. > >>> > >>> I have heard about it a little. Should we investigate adding it to our > >>> systems? > >>> Is there a way to suppress this warning? > >>> > >>> > >>> > >>> Brock Palen > >>> > >>> www.umich.edu/~brockp > >>> > >>> CAEN Advanced Computing > >>> XSEDE Campus Champion > >>> > >>> bro...@umich.edu > >>> > >>> (734)936-1985 > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> users mailing list > >>> > >>> us...@open-mpi.org > >>> > >>> Subscription: > >>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >>> > >>> Link to this post: > >>> http://www.open-mpi.org/community/lists/users/2014/08/25166.php > >>> > >>> > >>> _______________________________________________ > >>> users mailing list > >>> > >>> us...@open-mpi.org > >>> > >>> Subscription: > >>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >>> > >>> Link to this post: > >>> http://www.open-mpi.org/community/lists/users/2014/08/25169.php > >> > >> > >> _______________________________________________ > >> users mailing list > >> > >> us...@open-mpi.org > >> > >> Subscription: > >> http://www.open-mpi.org/mailman/listinfo.cgi/users > >> > >> Link to this post: > >> http://www.open-mpi.org/community/lists/users/2014/08/25170.php > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > > http://www.open-mpi.org/community/lists/users/2014/08/25172.php > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25174.php > > > > -- > > Kind Regards, > > M. > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25175.php
signature.asc
Description: Message signed with OpenPGP using GPGMail
