Please see the comments inline .
When I use cmd "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib,self,sm a_user.out" or "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib,self,sm --mca btl_openib_cpc_include rdmacm a_user.out", will print correct result, but use cmd "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib a_user.out", will print error mesg. I want to know: 1) When I want to use XRC, I must have a special IB switch? You don't have use special ib switches. 2) How can I use XRC in ompi,and on which situation the XRC feature will bring benifit? About XRC benefit you may check this paper : http://www.open-mpi.org/papers/euro-pvmmpi-2008-xrc/ How to use. By default openib btl use follow QPs configuration - P,128,256,192,128:S,2048,256,128,32:S,12288,256,128,32:S,65536,256,128,32 You may find some description about this format, here - https://svn.open-mpi.org/trac/ompi/ticket/1260 But , bottom line , if you want to switch default QP configuration from RC qps to XRC qps you may use follow command line parameter: --mca btl_openib_receive_queues X,128,256,192,128 X,2048,256,128,32 X,12288,256,128,32 X,65536,256,128,32 3) If this is only way to using XRC that using "-mca btl_openib_cpc_include xoob"? Actually this parameter does not enable XRC. Please see #2. 4) When using xoob, which transport and network protocol ompi will use? If you will use the parameter from #2 , the xoob cpc will be used automatically. I hope it helps. Pavel (Pasha) Shamis --- Application Performance Tools Group Computer Science and Math Division Oak Ridge National Laboratory
