btw, you may want to use latest mxm v3.1 which is part of hpcx package http://www.mellanox.com/products/hpcx
On Thu, Aug 28, 2014 at 4:10 AM, Brock Palen <bro...@umich.edu> wrote: > Brice, et al. > > Thanks a lot for this info. We are setting up new builds of OMPI 1.8.2 > with knem and mxm 3.0, > > If we have questions we will let you know. > > Brock Palen > www.umich.edu/~brockp > CAEN Advanced Computing > XSEDE Campus Champion > bro...@umich.edu > (734)936-1985 > > > > On Aug 27, 2014, at 12:44 PM, Brice Goglin <brice.gog...@inria.fr> wrote: > > > Hello Brock, > > > > Some people complained that giving world-wide access to a device file by > default might be bad if we ever find a security leak in the kernel module. > So I needed a better default. The rdma group is often used for OFED > devices, and OFED and KNEM users are often the same, so it was a good > compromise. > > > > There's no major issue with opening /dev/knem to everybody. A remote > process memory is only accessible if an attacker finds the corresponding > 64bit cookie. Only the memory buffer that was explicitly made readable > and/or writable can be accessed read and/or write through this cookie. And > recent KNEM releases also enforce by default that the attacker has the same > uid as the target process. > > > > Brice > > > > > > > > > > Le 27/08/2014 16:25, Brock Palen a écrit : > >> Is there any major issues letting all users use it by setting /dev/knem > to 666 ? It appears knem by default wants to only allow users of the rdma > group (if defined) to access knem. > >> > >> We are a generic provider and want everyone to be able to use it, just > feels strange to restrict it, so I am trying to understand why that is the > default. > >> > >> Brock Palen > >> > >> www.umich.edu/~brockp > >> > >> CAEN Advanced Computing > >> XSEDE Campus Champion > >> > >> bro...@umich.edu > >> > >> (734)936-1985 > >> > >> > >> > >> On Aug 27, 2014, at 10:15 AM, Alina Sklarevich > >> <ali...@dev.mellanox.co.il> > >> wrote: > >> > >> > >>> Hi, > >>> > >>> KNEM can improve the performance significantly for intra-node > communication and that's why MXM is using it. > >>> If you don't want to use it, you can suppress this warning by adding > the following to your command line after mpirun: > >>> -x MXM_LOG_LEVEL=error > >>> > >>> Alina. > >>> > >>> > >>> On Wed, Aug 27, 2014 at 4:28 PM, Brock Palen > >>> <bro...@umich.edu> > >>> wrote: > >>> We updated our ofed and started to rebuild our MPI builds with mxm > 3.0 . > >>> > >>> Now we get warnings bout knem > >>> > >>> [1409145437.578861] [flux-login1:31719:0] shm.c:65 MXM > WARN Could not open the KNEM device file at /dev/knem : No such file or > directory. Won't use knem. > >>> > >>> I have heard about it a little. Should we investigate adding it to > our systems? > >>> Is there a way to suppress this warning? > >>> > >>> > >>> > >>> Brock Palen > >>> > >>> www.umich.edu/~brockp > >>> > >>> CAEN Advanced Computing > >>> XSEDE Campus Champion > >>> > >>> bro...@umich.edu > >>> > >>> (734)936-1985 > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> users mailing list > >>> > >>> us...@open-mpi.org > >>> > >>> Subscription: > >>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >>> > >>> Link to this post: > >>> http://www.open-mpi.org/community/lists/users/2014/08/25166.php > >>> > >>> > >>> _______________________________________________ > >>> users mailing list > >>> > >>> us...@open-mpi.org > >>> > >>> Subscription: > >>> http://www.open-mpi.org/mailman/listinfo.cgi/users > >>> > >>> Link to this post: > >>> http://www.open-mpi.org/community/lists/users/2014/08/25169.php > >> > >> > >> _______________________________________________ > >> users mailing list > >> > >> us...@open-mpi.org > >> > >> Subscription: > >> http://www.open-mpi.org/mailman/listinfo.cgi/users > >> > >> Link to this post: > >> http://www.open-mpi.org/community/lists/users/2014/08/25170.php > > > > _______________________________________________ > > users mailing list > > us...@open-mpi.org > > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25172.php > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25174.php > -- Kind Regards, M.
