Re: [OMPI users] RoCE (IBoE) & OpenMPI

2011-02-18 Thread Michael Shuey
Per-node GID & SL settings == bad. Site-wide GID & SL settings == good. If this could be an MCA param (like btl_openib_ib_service_level) that'd be great - we already have a global config file of similar params. We'd definitely want the same N everywhere. -- Mike Shuey On Fri, Feb 18, 2011 at

Re: [OMPI users] RoCE (IBoE) & OpenMPI

2011-02-18 Thread Jeff Squyres
On Feb 18, 2011, at 1:39 PM, Michael Shuey wrote: > RoCE HCAs keep a GID table, like normal HCAs. Every time you bring up > a vlan interface, another entry gets automatically added to the table. > If I select one of these other GIDs, packets get a VLAN tag, and that > contains the necessary prior

Re: [OMPI users] RoCE (IBoE) & OpenMPI

2011-02-18 Thread Michael Shuey
It's a little different in RoCE. There's no subnet manager, so (as near as I can tell) you don't really have a subnet ID. Instead, the GID = GUID + VLAN tag (more or less). gid[0] has special bits in the VLAN tag section, to indicate that packets relating to this GID don't get a VLAN tag. Unfor

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Gus Correa
Hi Tena I am glad to see that it you're getting there! I'm following this thread all the time, but I can't really help on this one, for lack of any direct experience with EC2. It looks like now you've got the full scoop of advice from Jeff (OpenMPI side), Barnet and Ashley (EC2 side), which is

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Tena Sakai
Hi Ashley, > If you do this I would appreciate the chance to proof-read it before you go > public... Absolutely! I am a firm believer in two-heads-are-better-than- one concept. Regards, Tena On 2/18/11 1:29 AM, "Ashley Pittman" wrote: > > On 18 Feb 2011, at 09:09, Tena Sakai wrote: >> I h

[OMPI users] Beginner's question: why multiple sends or receives don't work?

2011-02-18 Thread Xianglong Kong
Hi, all, I’m an mpi newbie. I’m trying to connect two desktops in my office with each other using a crossing cable and implement a parallel code on them using MPI. Now, the two nodes can ssh to each other without password, and can successfully run the MPI “Hello world” code. However, when I tried

[OMPI users] nonblock alternative to MPI_Win_complete

2011-02-18 Thread Toon Knapen
(Probably this issue has been discussed at length before but unfortunately I did not find any threads (on this site or anywhere else) on this topic, if you are able to provide me with links to earlier discussions on this topic, please do not hesitate) Is there an alternative to MPI_Win_complete th

Re: [OMPI users] How are IP addresses determined?

2011-02-18 Thread Barnet Wagman
>Connection MethodProtocolFrom portTo port Source (IP > or group) > Alltcp > 0 65535intra ^ Yes, after I wrote yesterday, I realized there's no need to have the 'source' wide open. Glad to know that

Re: [OMPI users] RoCE (IBoE) & OpenMPI

2011-02-18 Thread Jeff Squyres
Greetings Mike. I'll answer today because Fri-Sat is the weekend in Israel (i.e., the MPI team at Mellanox won't see this until Sunday). I don't have a lot of experience with RoCE; do you need a different GUID or a different subnet ID? At least in IB, the GID = GUID + Subnet ID. The GUID sho

[OMPI users] RoCE (IBoE) & OpenMPI

2011-02-18 Thread Michael Shuey
I've been looking into OpenMPI's support for RoCE (Mellanox's recent Infiniband-over-Ethernet) lately. While it's promising, I've hit a snag: RoCE requires lossless ethernet, and on my switches the only way to guarantee this is with CoS. RoCE adapters cannot emit CoS priority tags unless the clie

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Ashley Pittman
On 18 Feb 2011, at 09:09, Tena Sakai wrote: > I had created a security group "intra." I opened ssh port from 0 to > 65535, and launched instances (I unleashed 2 at a time in a same > geography zone) each belonging to the group intra. So, here, ssh > is a security rule of a security group intra.

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Tena Sakai
Hi Gus, I am starting to see the light at the other end of the tunnel. As I wrote in reply to Jeff, it was not a ssh problem. It was a setting of user configurable firewall that Amazon calls security group. I need to expand my small tests to wider set, but I think I can do that. I will keep you

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Tena Sakai
Hi Jeff, I have chosen to call this thread "This must be ssh problem, but I can't figure out what it is..." It turns out that's really wrong! EC2 allows users to create what is called security group. A security group is made of one or more security rules, which is basically a port based firewal

Re: [OMPI users] How are IP addresses determined?

2011-02-18 Thread Tena Sakai
Hi Barnet, Thank you for your post. It was security group setting. Here’s my entry Connection MethodProtocolFrom portTo port Source (IP or group) Alltcp 0 65535intra I didn’t want to use 0.0.0.0/0 for s

Re: [OMPI users] This must be ssh problem, but I can't figure out what it is...

2011-02-18 Thread Tena Sakai
Hi Jeff, Thank you for your suggestions. I followed your steps verbatim. Unfortunately, there is a bit of problem. Here's what I did: [tsakai@vixen ec2]$ ssh -i $MYKEY tsa...@ec2-184-73-62-72.compute-1.amazonaws.com The authenticity of host 'ec2-184-73-62-72.compute-1.amazonaws.com (184.73.