>On Mar 14, 2014, at 10:16:34 AM,Jeff Squyres <jsquyres_at_[hidden]> wrote:
>
>>On Mar 14, 2014, at 10:11 AM, Ralph Castain <rhc_at_[hidden]> wrote:
>>
>>> 1. If specified '--mca btl tcp,self', which interface application will run
>>> on, use GigE adaper OR use the OpenFabrics interface in IP over IB mode
>>> (just like a high performance GigE adapter) ?
>>
>> Both - ip over ib looks just like an Ethernet adaptor
>
>
>To be clear: the TCP BTL will use all TCP interfaces (regardless of underlying
>physical transport). Your GigE adapter and your IP adapter both present IP
>interfaces to>the OS, and both support TCP. So the TCP BTL will use them,
>because it just sees the TCP/IP interfaces.
Thanks for your kindly input.
Please see if I have understood correctly
Assume there are two nework
Gigabit Ethernet
eth0-renamed : 192.168.[1-22].[1-14] / 255.255.192.0
InfiniBand network
ib0 : 172.20.[1-22].[1-4] / 255.255.0.0
1. If specified '--mca btl tcp,self
The control information ( such as setup and teardown ) are routed to and
passed by Gigabit Ethernet in TCP/IP mode
The MPI messages are routed to and passed by InfiniBand network in IP over
IB mode
On the same machine, the TCP lookback device will be used for passing
control and MPI messages
2. If specified '--mca btl tcp,self --mca btl_tcp_if_include ib0'
Both of control information ( such as setup and teardown ) and MPI
messages are routed to and passed by InfiniBand network in IP over IB mode
On the same machine, The TCP lookback device will be used for passing
control and MPI messages
3. If specified '--mca btl openib,self'
The control information ( such as setup and teardown ) are routed to and
passed by InfiniBand network in IP over IB mode
The MPI messages are routed to and passed by InfiniBand network in RDMA
mode
On the same machine, the TCP lookback device will be used for passing
control and MPI messages
4. If without specifiying any 'mca btl' parameters
The control information ( such as setup and teardown ) are routed to and
passed by Gigabit Ethernet in TCP/IP mode
The MPI messages are routed and passed by InfiniBand network in RDMA mode
On the same machine, the shared memory (sm) BTL will be used for control
and MPI passing messages
Appreciating your kindly input
Jianyu
