On Mar 10, 2006, at 2:24 AM, Troy Telford wrote:
On Mar 9, 2006, at 9:18 PM, Brian Barrett wrote:
On Mar 9, 2006, at 6:41 PM, Troy Telford wrote:
I've got a machine that has the following config:
Each node has two InfiniBand ports:
* The first port is on fabric 'a' with switches for 'a'
* The second port is on fabric 'b' with separate switches for 'b'
* The two fabrics are not shared ('a' and 'b' can't communicate
with one
another)
I believe that Open MPI is perfectly capable of stripeing over both
fabric
'a' and 'b', and IIRC, this is the default behavior.
Does Open MPI handle the case where Open MPI puts all of its
traffic on
the first IB port (ie. fabric 'a'), and leaves the second IB port
(ie.
fabric 'b') free for other uses (I'll use NFS as a humorous
example).
If so, is there any magic required to configure it thusly?
With mvapi, we don't have the functionality in place for the user to
specify which HCA port is used. The user can say that at most N HCA
ports should be used through the btl_mvapi_max_btls MCA parameter.
So in your case, if you ran Open MPI with:
mpirun -mca btl_mvapi_max_btls 1 -np X ./foobar
Only the first active port would be used for mvapi communication.
I'm not sure if this is enough for your needs or not.
So long as the second active port isn't touched by Open MPI, it
sounds just fine.
One thing, though -- You mention mvapi, which IIRC is the 1st
Generation IB stack. Is there similar functionality with the openib
btl (for the 2nd generation IB stack)?
It looks like we never added similar logic to the Open IB transport.
I'll pass your request on to the developer of our Open IB transport.
Given our timeframe for releasing Open MPI 1.0.2, it's doubtful any
change will make that release. But it should definitely be possible
to add such functionality in a future release.
Brian
