On 2/5/2016 2:38 AM, dpchoudh . wrote:
Dear all

This is a slightly off-topic post, and hopefully people won't mind
helping me out.

I have a very simple setup with two PCs, both with identical Chelsio
10GE iWARP adapter connected back-to-back.

With this setup, the TCP channel works fine (with MPI or otherwise).
But somehow, using RDMA semantics, the QP creation fails. This has
nothing to do with OpenMPI per se; using standard benchmarking tools
such as qperf also shows the same behavior.

I have set both the hard and soft limit on locked memory to
'unlimited' and have rebooted both PCs and have verified that 'ulimit
-l' shows 'unlimited' on both nodes. Yet, the QP creation fails. In
OpenMPI this shows up if I try to use the openib BTL. The OS (on both
nodes) is CentOS 7, if that matters.

What am I doing wrong?

Hey Durga,  Does rping work over the chelsio interfaces?

on one node run:  rping -s
on the other node run: rping -C10 -Vvca <peer-chelsio-ipaddress>

qperf is pretty old and doesn't work over iwarp. How about some of the other rdma tools? ib_write_bw for example?

Make sure you have these modules loaded:  iw_cxgb4 rdma_ucm
Make sure you have libibverbs, librdmacm, and libcxgb4 rpms installed

Does ibv_devinfo -l show the cxgb4_0 device?

steve

Reply via email to