Dave,
the QDR Infiniband uses the openib btl (by default :
btl_openib_exclusivity=1024)
i assume the RoCE 10Gbps card is using the tcp btl (by default :
btl_tcp_exclusivity=100)
that means that by default, when both openib and tcp btl could be used,
the tcp btl is discarded.
could you give a try
I ran some aggregate bandwidth tests between 2 hosts connected by
both QDR InfiniBand and RoCE enabled 10 Gbps Mellanox cards. The tests
measured the aggregate performance for 16 cores on one host communicating
with 16 on the second host. I saw the same performance as with the QDR
InfiniBand
