Re: [OMPI users] mixing MX and TCP

2007-06-12 Thread George Bosilca
Well, as expected this call is not documented ... and I get to it only with some help from Loic. On the version of MX I have tested it is not required to add the 3th and 4th arguments as I don't want to set anything. The NIC is already specified through the mx_btl->mx_endpoint isn't it ? geo

Re: [OMPI users] mixing MX and TCP

2007-06-11 Thread Reese Faucette
! if( (status = mx_get_info( mx_btl->mx_endpoint, MX_LINE_SPEED, !&nic_id, sizeof(nic_id), &value, sizeof(int))) != MX_SUCCESS ) { yes, a NIC ID is required for this call because a host may have multiple NICs with

Re: [OMPI users] mixing MX and TCP

2007-06-11 Thread George Bosilca
It's about using multiple network interfaces to exchange messages between a pair of hosts. The networks can be identical or not. george. On Jun 9, 2007, at 8:19 PM, Alex Tumanov wrote: forgive a trivial question, but what's a multi-rail? On 6/8/07, George Bosilca wrote: A fix for this p

Re: [OMPI users] mixing MX and TCP

2007-06-11 Thread Kees Verstoep
George Bosilca wrote: A fix for this problem is now available on the trunk. Please use any revision after 14963 and your problem will vanish [I hope!]. There are now some additional parameters which allow you to select which Myrinet network you want to use in the case there are several availab

Re: [OMPI users] mixing MX and TCP

2007-06-09 Thread Alex Tumanov
forgive a trivial question, but what's a multi-rail? On 6/8/07, George Bosilca wrote: A fix for this problem is now available on the trunk. Please use any revision after 14963 and your problem will vanish [I hope!]. There are now some additional parameters which allow you to select which Myrine

Re: [OMPI users] mixing MX and TCP

2007-06-08 Thread George Bosilca
A fix for this problem is now available on the trunk. Please use any revision after 14963 and your problem will vanish [I hope!]. There are now some additional parameters which allow you to select which Myrinet network you want to use in the case there are several available (--mca btl_mx_if

Re: [OMPI users] mixing MX and TCP

2007-06-05 Thread Jeff Squyres
FWIW, we do something similar in the openib BTL -- we use the subnet ID to determine if two IB ports are connected (we have the rule in OMPI that physically disconnected subnets must have different ID's -- this is more stringent than the IB spec calls for). See: http://www.open-mpi.org/faq

Re: [OMPI users] mixing MX and TCP

2007-06-01 Thread Reese Faucette
Just to brainstorm on this a little - the two different clusters will have different "mapper IDs", and this can be learned via the attached code snippet. As long as fma is the mapper (as opposed the the older, deprecated "gm_mapper" or "mx_mapper"), then Myrinet topology rules ensure that NIC 0

Re: [OMPI users] mixing MX and TCP

2007-06-01 Thread George Bosilca
Well, we are aware of this problem, but to be honest I was ready to bet that nobody will use a cluster of cluster with Myrinet and TCP ... so it was in a low priority TODO list. The problem is the routing table of the MX device. The MX BTL is unable to identify in a unique manner that there

Re: [OMPI users] mixing MX and TCP

2007-06-01 Thread Christian Kauhaus
Kees Verstoep : >I am currently experimenting with OpenMPI in a multi-cluster setting >where each cluster has its private Myri-10G/MX network besides TCP. Very interesting topic. :) >I see MX rather than tcp-level connections between clusters being >tried, which across clusters fails in mx_connec

[OMPI users] mixing MX and TCP

2007-05-31 Thread Kees Verstoep
Hi, I am currently experimenting with OpenMPI in a multi-cluster setting where each cluster has its private Myri-10G/MX network besides TCP. Somehow I was under the assumption that OpenMPI would dynamically find out the details of this configuration, and use MX where possible (i.e., intra cluster