Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-28 Thread Brock Palen
Interesting, we are using 3.0 that is in MOFED, and that is also what is on the MXM download site. Kinda confusing. Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion bro...@umich.edu (734)936-1985 On Aug 28, 2014, at 2:12 AM, Mike Dubman wrote: > btw, you may w

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-28 Thread Mike Dubman
btw, you may want to use latest mxm v3.1 which is part of hpcx package http://www.mellanox.com/products/hpcx On Thu, Aug 28, 2014 at 4:10 AM, Brock Palen wrote: > Brice, et al. > > Thanks a lot for this info. We are setting up new builds of OMPI 1.8.2 > with knem and mxm 3.0, > > If we have qu

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-27 Thread Brock Palen
Brice, et al. Thanks a lot for this info. We are setting up new builds of OMPI 1.8.2 with knem and mxm 3.0, If we have questions we will let you know. Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion bro...@umich.edu (734)936-1985 On Aug 27, 2014, at 12:44 PM,

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-27 Thread Brice Goglin
Hello Brock, Some people complained that giving world-wide access to a device file by default might be bad if we ever find a security leak in the kernel module. So I needed a better default. The rdma group is often used for OFED devices, and OFED and KNEM users are often the same, so it was a good

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-27 Thread Alina Sklarevich
I'm not sure why this is the default but in your case you should set the permissions to 666 to use it. On Wed, Aug 27, 2014 at 5:25 PM, Brock Palen wrote: > Is there any major issues letting all users use it by setting /dev/knem to > 666 ? It appears knem by default wants to only allow users o

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-27 Thread Brock Palen
Is there any major issues letting all users use it by setting /dev/knem to 666 ? It appears knem by default wants to only allow users of the rdma group (if defined) to access knem. We are a generic provider and want everyone to be able to use it, just feels strange to restrict it, so I am tr

Re: [OMPI users] mxm 3.0 and knem warnings

2014-08-27 Thread Alina Sklarevich
Hi, KNEM can improve the performance significantly for intra-node communication and that's why MXM is using it. If you don't want to use it, you can suppress this warning by adding the following to your command line after mpirun: -x MXM_LOG_LEVEL=error Alina. On Wed, Aug 27, 2014 at 4:28 PM, Br