On 2/5/2016 2:38 AM, dpchoudh . wrote:
Dear all
This is a slightly off-topic post, and hopefully people won't mind
helping me out.
I have a very simple setup with two PCs, both with identical Chelsio
10GE iWARP adapter connected back-to-back.
With this setup, the TCP channel works fine (with MPI or otherwise).
But somehow, using RDMA semantics, the QP creation fails. This has
nothing to do with OpenMPI per se; using standard benchmarking tools
such as qperf also shows the same behavior.
I have set both the hard and soft limit on locked memory to
'unlimited' and have rebooted both PCs and have verified that 'ulimit
-l' shows 'unlimited' on both nodes. Yet, the QP creation fails. In
OpenMPI this shows up if I try to use the openib BTL. The OS (on both
nodes) is CentOS 7, if that matters.
What am I doing wrong?
Hey Durga, Does rping work over the chelsio interfaces?
on one node run: rping -s
on the other node run: rping -C10 -Vvca <peer-chelsio-ipaddress>
qperf is pretty old and doesn't work over iwarp. How about some of the
other rdma tools? ib_write_bw for example?
Make sure you have these modules loaded: iw_cxgb4 rdma_ucm
Make sure you have libibverbs, librdmacm, and libcxgb4 rpms installed
Does ibv_devinfo -l show the cxgb4_0 device?
steve
