Am Fri, 24 Nov 2017 14:50:28 +0000 (UTC) Hans Petter Selasky <hsela...@freebsd.org> schrieb:
> Author: hselasky > Date: Fri Nov 24 14:50:28 2017 > New Revision: 326169 > URL: https://svnweb.freebsd.org/changeset/base/326169 > > Log: > RoCE/infiniband upgrade to Linux v4.9 for kernel and userspace. > > This commit merges projects/bsd_rdma_4_9 to head. > > List of kernel sources used: > ============================ > > 1) kernel sources were cloned from git://github.com/torvalds/linux.git > Top commit 69973b830859bc6529a7a0468ba0d80ee5117826 - tag: v4.9, linux-4.9 > > 2) krping was cloned from https://github.com/larrystevenwise/krping > Top commit 292a2f1abf0348285e678a82264740d52e4dcfe4 > > List of userspace sources used: > =============================== > > 1) rdma-core was cloned from https://github.com/linux-rdma/rdma-core.git > Top commit d65138ef93af30b3ea249f3a84aa6a24ba7f8a75 > > 2) OpenSM was cloned from git://git.openfabrics.org/~halr/opensm.git > Top commit 85f841cf209f791c89a075048a907020e924528d > > 3) libibmad was cloned from git://git.openfabrics.org/~iraweiny/libibmad.git > Tag 1.3.13 with some additional patches from Mellanox. > > 4) infiniband-diags was cloned from > git://git.openfabrics.org/~iraweiny/infiniband-diags.git Tag 1.6.7 with some > additional > patches from Mellanox. > NOTES: > ====== > > 1) The mthca driver has been removed in kernel and in userspace. > 2) All GPLv2 only sources have been removed and where applicable > rewritten from scratch under a BSD license. > 3) List of fully supported drivers in userspace and kernel: > a) iw_cxgbe (Chelsio) > b) mlx4ib (Mellanox) > c) mlx5ib (Mellanox) > 4) WITH_OFED=YES is still required by make in order to build > OFED userspace and kernel code. > 5) Full support has been added for routable RoCE, RoCE v2. > > Sponsored by: Mellanox Technologies > > Added: > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_gsi.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/dev/mlx5/mlx5_ib/mlx5_ib_gsi.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_virt.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/dev/mlx5/mlx5_ib/mlx5_ib_virt.c > head/sys/ofed/drivers/infiniband/core/ib_addr.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_addr.c > head/sys/ofed/drivers/infiniband/core/ib_agent.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_agent.c > head/sys/ofed/drivers/infiniband/core/ib_cache.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_cache.c > head/sys/ofed/drivers/infiniband/core/ib_cm.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_cm.c > head/sys/ofed/drivers/infiniband/core/ib_cma.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_cma.c > head/sys/ofed/drivers/infiniband/core/ib_cq.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_cq.c > head/sys/ofed/drivers/infiniband/core/ib_device.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_device.c > head/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c > head/sys/ofed/drivers/infiniband/core/ib_iwcm.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_iwcm.c > head/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c > head/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c > head/sys/ofed/drivers/infiniband/core/ib_mad.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_mad.c > head/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c > head/sys/ofed/drivers/infiniband/core/ib_multicast.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_multicast.c > head/sys/ofed/drivers/infiniband/core/ib_packer.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_packer.c > head/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c > head/sys/ofed/drivers/infiniband/core/ib_sa_query.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_sa_query.c > head/sys/ofed/drivers/infiniband/core/ib_smi.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_smi.c > head/sys/ofed/drivers/infiniband/core/ib_sysfs.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_sysfs.c > head/sys/ofed/drivers/infiniband/core/ib_ucm.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_ucm.c > head/sys/ofed/drivers/infiniband/core/ib_ucma.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_ucma.c > head/sys/ofed/drivers/infiniband/core/ib_ud_header.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_ud_header.c > head/sys/ofed/drivers/infiniband/core/ib_umem.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_umem.c > head/sys/ofed/drivers/infiniband/core/ib_umem_odp.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_umem_odp.c > head/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c > head/sys/ofed/drivers/infiniband/core/ib_user_mad.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_user_mad.c > head/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c > head/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c > head/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c > head/sys/ofed/drivers/infiniband/core/ib_verbs.c > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/ib_verbs.c > head/sys/ofed/drivers/infiniband/core/iwpm_util.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/iwpm_util.h > head/sys/ofed/drivers/infiniband/core/opa_smi.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/drivers/infiniband/core/opa_smi.h > head/sys/ofed/include/rdma/ib.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/ib.h > head/sys/ofed/include/rdma/ib_hdrs.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/ib_hdrs.h > head/sys/ofed/include/rdma/ib_umem_odp.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/ib_umem_odp.h > head/sys/ofed/include/rdma/iw_portmap.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/iw_portmap.h > head/sys/ofed/include/rdma/opa_port_info.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/opa_port_info.h > head/sys/ofed/include/rdma/opa_smi.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/opa_smi.h > head/sys/ofed/include/rdma/rdma_vt.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/rdma_vt.h > head/sys/ofed/include/rdma/rdmavt_cq.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/rdmavt_cq.h > head/sys/ofed/include/rdma/rdmavt_mr.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/rdmavt_mr.h > head/sys/ofed/include/rdma/rdmavt_qp.h > - copied unchanged from r326168, > projects/bsd_rdma_4_9/sys/ofed/include/rdma/rdmavt_qp.h > head/sys/ofed/include/uapi/ > - copied from r326168, projects/bsd_rdma_4_9/sys/ofed/include/uapi/ > Replaced: > head/contrib/ofed/ > - copied from r326168, projects/bsd_rdma_4_9/contrib/ofed/ > Deleted: > head/sys/dev/mlx4/mlx4_ib/mlx4_exp.h > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_exp.c > head/sys/dev/mlx4/mlx4_ib/user.h > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_roce.c > head/sys/dev/mlx5/mlx5_ib/user.h > head/sys/modules/mthca/ > head/sys/ofed/drivers/infiniband/Kconfig > head/sys/ofed/drivers/infiniband/Makefile > head/sys/ofed/drivers/infiniband/core/addr.c > head/sys/ofed/drivers/infiniband/core/agent.c > head/sys/ofed/drivers/infiniband/core/cache.c > head/sys/ofed/drivers/infiniband/core/cm.c > head/sys/ofed/drivers/infiniband/core/cma.c > head/sys/ofed/drivers/infiniband/core/device.c > head/sys/ofed/drivers/infiniband/core/fmr_pool.c > head/sys/ofed/drivers/infiniband/core/iwcm.c > head/sys/ofed/drivers/infiniband/core/mad.c > head/sys/ofed/drivers/infiniband/core/mad_rmpp.c > head/sys/ofed/drivers/infiniband/core/multicast.c > head/sys/ofed/drivers/infiniband/core/packer.c > head/sys/ofed/drivers/infiniband/core/peer_mem.c > head/sys/ofed/drivers/infiniband/core/sa_query.c > head/sys/ofed/drivers/infiniband/core/smi.c > head/sys/ofed/drivers/infiniband/core/sysfs.c > head/sys/ofed/drivers/infiniband/core/ucm.c > head/sys/ofed/drivers/infiniband/core/ucma.c > head/sys/ofed/drivers/infiniband/core/ud_header.c > head/sys/ofed/drivers/infiniband/core/umem.c > head/sys/ofed/drivers/infiniband/core/user_mad.c > head/sys/ofed/drivers/infiniband/core/uverbs_cmd.c > head/sys/ofed/drivers/infiniband/core/uverbs_main.c > head/sys/ofed/drivers/infiniband/core/uverbs_marshall.c > head/sys/ofed/drivers/infiniband/core/verbs.c > head/sys/ofed/drivers/infiniband/debug/Makefile > head/sys/ofed/drivers/infiniband/debug/memtrack.c > head/sys/ofed/drivers/infiniband/debug/memtrack.h > head/sys/ofed/drivers/infiniband/debug/mtrack.h > head/sys/ofed/drivers/infiniband/hw/ > head/sys/ofed/drivers/infiniband/ulp/ipoib/Kconfig > head/sys/ofed/drivers/infiniband/util/Kconfig > head/sys/ofed/include/rdma/ib_peer_mem.h > head/sys/ofed/include/rdma/ib_user_cm.h > head/sys/ofed/include/rdma/ib_user_mad.h > head/sys/ofed/include/rdma/ib_user_sa.h > head/sys/ofed/include/rdma/ib_user_verbs.h > head/sys/ofed/include/rdma/ib_user_verbs_exp.h > head/sys/ofed/include/rdma/ib_verbs_exp.h > head/sys/ofed/include/rdma/peer_mem.h > head/sys/ofed/include/rdma/rdma_user_cm.h > head/sys/ofed/include/rdma/sdp_socket.h > Modified: > head/Makefile.inc1 > head/share/mk/bsd.libnames.mk > head/share/mk/src.libnames.mk > head/sys/amd64/conf/NOTES > head/sys/conf/files > head/sys/conf/kern.pre.mk > head/sys/contrib/rdma/krping/krping.c > head/sys/contrib/rdma/krping/krping.h > head/sys/contrib/rdma/krping/krping_dev.c > head/sys/dev/cxgbe/adapter.h > head/sys/dev/cxgbe/common/common.h > head/sys/dev/cxgbe/common/t4_hw.c > head/sys/dev/cxgbe/iw_cxgbe/cm.c > head/sys/dev/cxgbe/iw_cxgbe/cq.c > head/sys/dev/cxgbe/iw_cxgbe/device.c > head/sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h > head/sys/dev/cxgbe/iw_cxgbe/mem.c > head/sys/dev/cxgbe/iw_cxgbe/provider.c > head/sys/dev/cxgbe/iw_cxgbe/qp.c > head/sys/dev/cxgbe/iw_cxgbe/t4.h > head/sys/dev/cxgbe/iw_cxgbe/user.h > head/sys/dev/cxgbe/offload.h > head/sys/dev/cxgbe/t4_main.c > head/sys/dev/iser/icl_iser.c > head/sys/dev/iser/icl_iser.h > head/sys/dev/iser/iser_memory.c > head/sys/dev/iser/iser_verbs.c > head/sys/dev/mlx4/device.h > head/sys/dev/mlx4/mlx4_ib/mlx4_ib.h > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_alias_GUID.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_cm.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_cq.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_mad.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_main.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_mcg.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_mr.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_qp.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_srq.c > head/sys/dev/mlx4/mlx4_ib/mlx4_ib_sysfs.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib.h > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_ah.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_cq.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mad.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mem.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mr.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c > head/sys/dev/mlx5/mlx5_ib/mlx5_ib_srq.c > head/sys/dev/mlx5/qp.h > head/sys/i386/conf/NOTES > head/sys/modules/Makefile > head/sys/modules/cxgbe/iw_cxgbe/Makefile > head/sys/modules/ibcore/Makefile > head/sys/modules/ipoib/Makefile > head/sys/modules/iser/Makefile > head/sys/modules/mlx4/Makefile > head/sys/modules/mlx4ib/Makefile > head/sys/modules/mlx5ib/Makefile > head/sys/modules/rdma/krping/Makefile > head/sys/ofed/drivers/infiniband/core/agent.h > head/sys/ofed/drivers/infiniband/core/cm_msgs.h > head/sys/ofed/drivers/infiniband/core/core_priv.h > head/sys/ofed/drivers/infiniband/core/iwcm.h > head/sys/ofed/drivers/infiniband/core/mad_priv.h > head/sys/ofed/drivers/infiniband/core/smi.h > head/sys/ofed/drivers/infiniband/core/uverbs.h > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ib.c > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_multicast.c > head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_verbs.c > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp.h > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_cma.c > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_main.c > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_proc.c > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_rx.c > head/sys/ofed/drivers/infiniband/ulp/sdp/sdp_tx.c > head/sys/ofed/include/rdma/ib_addr.h > head/sys/ofed/include/rdma/ib_cache.h > head/sys/ofed/include/rdma/ib_cm.h > head/sys/ofed/include/rdma/ib_mad.h > head/sys/ofed/include/rdma/ib_pack.h > head/sys/ofed/include/rdma/ib_pma.h > head/sys/ofed/include/rdma/ib_sa.h > head/sys/ofed/include/rdma/ib_smi.h > head/sys/ofed/include/rdma/ib_umem.h > head/sys/ofed/include/rdma/ib_verbs.h > head/sys/ofed/include/rdma/iw_cm.h > head/sys/ofed/include/rdma/rdma_cm.h > head/targets/pseudo/userland/lib/Makefile.depend > Directory Properties: > head/ (props changed) > head/MAINTAINERS (props changed) > head/cddl/ (props changed) > head/cddl/contrib/opensolaris/ (props changed) > head/cddl/contrib/opensolaris/cmd/zdb/ (props changed) > head/cddl/contrib/opensolaris/cmd/zfs/ (props changed) > head/cddl/contrib/opensolaris/lib/libzfs/ (props changed) > head/contrib/atf/ (props changed) > head/contrib/binutils/ (props changed) > head/contrib/blacklist/ (props changed) > head/contrib/bmake/ (props changed) > head/contrib/byacc/ (props changed) > head/contrib/compiler-rt/ (props changed) > head/contrib/dma/ (props changed) > head/contrib/elftoolchain/ (props changed) > head/contrib/elftoolchain/ar/ (props changed) > head/contrib/elftoolchain/elfdump/ (props changed) > head/contrib/file/ (props changed) > head/contrib/gcc/ (props changed) > head/contrib/ipfilter/ (props changed) > head/contrib/less/ (props changed) > head/contrib/libarchive/ (props changed) > head/contrib/libc++/ (props changed) > head/contrib/libc-vis/ (props changed) > head/contrib/libpcap/ (props changed) > head/contrib/libstdc++/ (props changed) > head/contrib/libxo/ (props changed) > head/contrib/llvm/ (props changed) > head/contrib/llvm/projects/libunwind/ (props changed) > head/contrib/llvm/tools/clang/ (props changed) > head/contrib/llvm/tools/lld/ (props changed) > head/contrib/llvm/tools/lldb/ (props changed) > head/contrib/mdocml/ (props changed) > head/contrib/netbsd-tests/ (props changed) > head/contrib/one-true-awk/ (props changed) > head/contrib/openbsm/ (props changed) > head/contrib/openpam/ (props changed) > head/contrib/pjdfstest/ (props changed) > head/contrib/sqlite3/ (props changed) > head/contrib/subversion/ (props changed) > head/contrib/tcpdump/ (props changed) > head/contrib/top/ (props changed) > head/contrib/tzdata/ (props changed) > head/contrib/unbound/ (props changed) > head/contrib/wpa/ (props changed) > head/contrib/xz/ (props changed) > head/contrib/zlib/ (props changed) > head/crypto/heimdal/ (props changed) > head/crypto/openssh/ (props changed) > head/crypto/openssl/ (props changed) > head/gnu/lib/ (props changed) > head/gnu/usr.bin/binutils/ (props changed) > head/gnu/usr.bin/cc/cc_tools/ (props changed) > head/gnu/usr.bin/gdb/ (props changed) > head/lib/libc/locale/ascii.c (props changed) > head/lib/libedit/ (props changed) > head/sys/amd64/amd64/efirt_machdep.c (props changed) > head/sys/cddl/contrib/opensolaris/ (props changed) > head/sys/contrib/dev/acpica/ (props changed) > head/sys/contrib/ena-com/ (props changed) > head/sys/contrib/ipfilter/ (props changed) > head/sys/contrib/octeon-sdk/ (props changed) > head/sys/contrib/zstd/ (props changed) > head/sys/dev/efidev/efirt.c (props changed) > head/sys/gnu/dts/arm/ (props changed) > head/sys/gnu/dts/include/ (props changed) > head/usr.sbin/bhyve/bhyvegc.c (props changed) > head/usr.sbin/bhyve/pci_fbuf.c (props changed) > head/usr.sbin/bhyve/pci_xhci.c (props changed) > head/usr.sbin/bhyve/rfb.c (props changed) > head/usr.sbin/bhyve/rfb.h (props changed) > head/usr.sbin/bhyve/vga.c (props changed) > > Modified: head/Makefile.inc1 > ============================================================================== > --- head/Makefile.inc1 Fri Nov 24 14:29:32 2017 (r326168) > +++ head/Makefile.inc1 Fri Nov 24 14:50:28 2017 (r326169) > @@ -2410,16 +2410,24 @@ _lib_libradius= lib/libradius > .endif > > .if ${MK_OFED} != "no" > -_ofed_lib= contrib/ofed/usr.lib > -_prebuild_libs+= contrib/ofed/usr.lib/libosmcomp > -_prebuild_libs+= contrib/ofed/usr.lib/libopensm > -_prebuild_libs+= contrib/ofed/usr.lib/libibcommon > -_prebuild_libs+= contrib/ofed/usr.lib/libibverbs > -_prebuild_libs+= contrib/ofed/usr.lib/libibumad > +# > +# The OFED libraries are built in four steps > +# as reflected below, due to interdependencies. > +# > +# NOTE: Depending on contrib/ofed/include is only needed for > +# the lib32 compat build. > +# > +_ofed_lib= \ > +contrib/ofed/include \ > +contrib/ofed/usr.lib/0 \ > +contrib/ofed/usr.lib/1 \ > +contrib/ofed/usr.lib/2 \ > +contrib/ofed/usr.lib/3 > > -contrib/ofed/usr.lib/libopensm__L: lib/libthr__L > -contrib/ofed/usr.lib/libosmcomp__L: lib/libthr__L > -contrib/ofed/usr.lib/libibumad__L: contrib/ofed/usr.lib/libibcommon__L > +contrib/ofed/usr.lib/0__L: contrib/ofed/include__L lib/libthr__L > +contrib/ofed/usr.lib/1__L: contrib/ofed/usr.lib/0__L > +contrib/ofed/usr.lib/2__L: contrib/ofed/usr.lib/1__L > +contrib/ofed/usr.lib/3__L: contrib/ofed/usr.lib/2__L > .endif > > .if ${MK_CASPER} != "no" > > Modified: head/share/mk/bsd.libnames.mk > ============================================================================== > --- head/share/mk/bsd.libnames.mk Fri Nov 24 14:29:32 2017 > (r326168) > +++ head/share/mk/bsd.libnames.mk Fri Nov 24 14:50:28 2017 > (r326169) > @@ -82,9 +82,8 @@ LIBHEIMNTLM?= ${LIBDESTDIR}${LIBDIR_BASE}/libheimntlm. > LIBHEIMSQLITE?= ${LIBDESTDIR}${LIBDIR_BASE}/libheimsqlite.a > LIBHX509?= ${LIBDESTDIR}${LIBDIR_BASE}/libhx509.a > LIBIBCM?= ${LIBDESTDIR}${LIBDIR_BASE}/libibcm.a > -LIBIBCOMMON?= ${LIBDESTDIR}${LIBDIR_BASE}/libibcommon.a > LIBIBMAD?= ${LIBDESTDIR}${LIBDIR_BASE}/libibmad.a > -LIBIBSDP?= ${LIBDESTDIR}${LIBDIR_BASE}/libibsdp.a > +LIBIBNETDISC?= ${LIBDESTDIR}${LIBDIR_BASE}/libibnetdisc.a > LIBIBUMAD?= ${LIBDESTDIR}${LIBDIR_BASE}/libibumad.a > LIBIBVERBS?= ${LIBDESTDIR}${LIBDIR_BASE}/libibverbs.a > LIBIFCONFIG?= ${LIBDESTDIR}${LIBDIR_BASE}/libifconfig.a > @@ -108,9 +107,9 @@ LIBMEMSTAT?= ${LIBDESTDIR}${LIBDIR_BASE}/libmemstat.a > LIBMENU?= ${LIBDESTDIR}${LIBDIR_BASE}/libmenu.a > LIBMILTER?= ${LIBDESTDIR}${LIBDIR_BASE}/libmilter.a > LIBMLX4?= ${LIBDESTDIR}${LIBDIR_BASE}/libmlx4.a > +LIBMLX5?= ${LIBDESTDIR}${LIBDIR_BASE}/libmlx5.a > LIBMP?= ${LIBDESTDIR}${LIBDIR_BASE}/libmp.a > LIBMT?= ${LIBDESTDIR}${LIBDIR_BASE}/libmt.a > -LIBMTHCA?= ${LIBDESTDIR}${LIBDIR_BASE}/libmthca.a > LIBNANDFS?= ${LIBDESTDIR}${LIBDIR_BASE}/libnandfs.a > LIBNCURSES?= ${LIBDESTDIR}${LIBDIR_BASE}/libncurses.a > LIBNCURSESW?= ${LIBDESTDIR}${LIBDIR_BASE}/libncursesw.a > > Modified: head/share/mk/src.libnames.mk > ============================================================================== > --- head/share/mk/src.libnames.mk Fri Nov 24 14:29:32 2017 > (r326168) > +++ head/share/mk/src.libnames.mk Fri Nov 24 14:50:28 2017 > (r326169) > @@ -194,18 +194,16 @@ _LIBRARIES+= \ > _LIBRARIES+= \ > cxgb4 \ > ibcm \ > - ibcommon \ > ibmad \ > - ibsdp \ > + ibnetdisc \ > ibumad \ > ibverbs \ > mlx4 \ > - mthca \ > - opensm \ > - osmcomp \ > - osmvendor \ > + mlx5 \ > rdmacm \ > - > + osmcomp \ > + opensm \ > + osmvendor > .endif > > # Each library's LIBADD needs to be duplicated here for static linkage of > @@ -332,17 +330,21 @@ _DP_zfs= md pthread umem util uutil m nvpair avl > bsdxm > zfs_core > _DP_zfs_core= nvpair > _DP_zpool= md pthread z nvpair avl umem > + > +# OFED support > .if ${MK_OFED} != "no" > _DP_cxgb4= ibverbs pthread > _DP_ibcm= ibverbs > -_DP_ibmad= ibcommon ibumad > -_DP_ibumad= ibcommon > +_DP_ibmad= ibumad > +_DP_ibnetdisc= osmcomp ibmad ibumad > +_DP_ibumad= > +_DP_ibverbs= > _DP_mlx4= ibverbs pthread > -_DP_mthca= ibverbs pthread > -_DP_opensm= pthread > -_DP_osmcomp= pthread > -_DP_osmvendor= ibumad opensm osmcomp pthread > +_DP_mlx5= ibverbs pthread > _DP_rdmacm= ibverbs > +_DP_osmcomp= pthread > +_DP_opensm= pthread > +_DP_osmvendor= ibumad pthread > .endif > > # Define special cases > @@ -483,19 +485,21 @@ LIBUUTILDIR= ${OBJTOP}/cddl/lib/libuutil > LIBZFSDIR= ${OBJTOP}/cddl/lib/libzfs > LIBZFS_COREDIR= ${OBJTOP}/cddl/lib/libzfs_core > LIBZPOOLDIR= ${OBJTOP}/cddl/lib/libzpool > -LIBCXGB4DIR= ${OBJTOP}/contrib/ofed/usr.lib/libcxgb4 > -LIBIBCMDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibcm > -LIBIBCOMMONDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibcommon > -LIBIBMADDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibmad > -LIBIBUMADDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibumad > -LIBIBVERBSDIR= ${OBJTOP}/contrib/ofed/usr.lib/libibverbs > -LIBMLX4DIR= ${OBJTOP}/contrib/ofed/usr.lib/libmlx4 > -LIBMTHCADIR= ${OBJTOP}/contrib/ofed/usr.lib/libmthca > -LIBOPENSMDIR= ${OBJTOP}/contrib/ofed/usr.lib/libopensm > -LIBOSMCOMPDIR= ${OBJTOP}/contrib/ofed/usr.lib/libosmcomp > -LIBOSMVENDORDIR= ${OBJTOP}/contrib/ofed/usr.lib/libosmvendor > -LIBRDMACMDIR= ${OBJTOP}/contrib/ofed/usr.lib/librdmacm > -LIBIBSDPDIR= ${OBJTOP}/contrib/ofed/usr.lib/libsdp > + > +# OFED support > +LIBCXGB4DIR= ${OBJTOP}/contrib/ofed/libcxgb4 > +LIBIBCMDIR= ${OBJTOP}/contrib/ofed/libibcm > +LIBIBMADDIR= ${OBJTOP}/contrib/ofed/libibmad > +LIBIBNETDISCDIR=${OBJTOP}/contrib/ofed/libibnetdisc > +LIBIBUMADDIR= ${OBJTOP}/contrib/ofed/libibumad > +LIBIBVERBSDIR= ${OBJTOP}/contrib/ofed/libibverbs > +LIBMLX4DIR= ${OBJTOP}/contrib/ofed/libmlx4 > +LIBMLX5DIR= ${OBJTOP}/contrib/ofed/libmlx5 > +LIBRDMACMDIR= ${OBJTOP}/contrib/ofed/librdmacm > +LIBOSMCOMPDIR= ${OBJTOP}/contrib/ofed/opensm/complib > +LIBOPENSMDIR= ${OBJTOP}/contrib/ofed/opensm/libopensm > +LIBOSMVENDORDIR=${OBJTOP}/contrib/ofed/opensm/libvendor > + > LIBDIALOGDIR= ${OBJTOP}/gnu/lib/libdialog > LIBGCOVDIR= ${OBJTOP}/gnu/lib/libgcov > LIBGOMPDIR= ${OBJTOP}/gnu/lib/libgomp > > Modified: head/sys/amd64/conf/NOTES > ============================================================================== > --- head/sys/amd64/conf/NOTES Fri Nov 24 14:29:32 2017 (r326168) > +++ head/sys/amd64/conf/NOTES Fri Nov 24 14:50:28 2017 (r326169) > @@ -317,7 +317,6 @@ options DRM_DEBUG # Include debug printfs (slow) > # ixlv: Intel XL710 40Gbe VF PCIE Ethernet > # mlx4ib: Mellanox ConnectX HCA InfiniBand > # mlx4en: Mellanox ConnectX HCA Ethernet > -# mthca: Mellanox HCA InfiniBand > # nfe: nVidia nForce MCP on-board Ethernet Networking (BSD open source) > # sfxge: Solarflare SFC9000 family 10Gb Ethernet adapters > # vmx: VMware VMXNET3 Ethernet (BSD open source) > @@ -338,7 +337,6 @@ device ixlv # Intel XL710 40Gbe VF > PCIE Ethernet device mlx4 # Shared code module between IB > and > Ethernet device mlx4ib # Mellanox ConnectX HCA InfiniBand > device mlx4en # Mellanox ConnectX HCA Ethernet > -device mthca # Mellanox HCA InfiniBand > device nfe # nVidia nForce MCP on-board Ethernet > device sfxge # Solarflare SFC9000 10Gb Ethernet > device vmx # VMware VMXNET3 Ethernet > > Modified: head/sys/conf/files > ============================================================================== > --- head/sys/conf/files Fri Nov 24 14:29:32 2017 (r326168) > +++ head/sys/conf/files Fri Nov 24 14:50:28 2017 (r326169) > @@ -4453,58 +4453,62 @@ compat/linuxkpi/common/src/linux_work.c > optional > comp compile-with "${LINUXKPI_C}" > > # OpenFabrics Enterprise Distribution (Infiniband) > -ofed/drivers/infiniband/core/addr.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/agent.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/cache.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -# XXX Mad.c must be ordered before cm.c for sysinit sets to occur in > -# the correct order. > -ofed/drivers/infiniband/core/mad.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/cm.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/ > -Wno-unused-function" -ofed/drivers/infiniband/core/cma.c optional > ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/device.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/fmr_pool.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/iwcm.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/mad_rmpp.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/multicast.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/packer.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/peer_mem.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/sa_query.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/smi.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/sysfs.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/ucm.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/ucma.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/ud_header.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/umem.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/user_mad.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/uverbs_cmd.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/uverbs_main.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/uverbs_marshall.c optional ofed > \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > -ofed/drivers/infiniband/core/verbs.c optional ofed \ > - compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/core/" > +ofed/drivers/infiniband/core/ib_addr.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_agent.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_cache.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_cm.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_cma.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_cq.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_device.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_fmr_pool.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_iwcm.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_iwpm_msg.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_iwpm_util.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_mad.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_mad_rmpp.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_multicast.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_packer.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_sa_query.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_smi.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_sysfs.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_ucm.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_ucma.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_ud_header.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_umem.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_user_mad.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_uverbs_cmd.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_uverbs_main.c optional ofed > \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_uverbs_marshall.c optional ofed \ > + compile-with "${OFED_C}" > +ofed/drivers/infiniband/core/ib_verbs.c optional ofed > \ > + compile-with "${OFED_C}" > > ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c optional ipoib \ > compile-with "${OFED_C} -I$S/ofed/drivers/infiniband/ulp/ipoib/" > @@ -4550,8 +4554,6 @@ dev/mlx4/mlx4_ib/mlx4_ib_mad.c optional > mlx4ib pci o compile-with "${OFED_C}" > dev/mlx4/mlx4_ib/mlx4_ib_main.c optional mlx4ib pci > ofed \ > compile-with "${OFED_C}" > -dev/mlx4/mlx4_ib/mlx4_ib_exp.c optional mlx4ib pci > ofed \ > - compile-with "${OFED_C}" > dev/mlx4/mlx4_ib/mlx4_ib_mr.c optional mlx4ib pci > ofed \ > compile-with "${OFED_C}" > dev/mlx4/mlx4_ib/mlx4_ib_qp.c optional mlx4ib pci > ofed \ > @@ -4623,6 +4625,8 @@ dev/mlx5/mlx5_ib/mlx5_ib_cq.c optional > mlx5ib pci of compile-with "${OFED_C}" > dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c optional mlx5ib pci ofed \ > compile-with "${OFED_C}" > +dev/mlx5/mlx5_ib/mlx5_ib_gsi.c optional mlx5ib pci > ofed \ > + compile-with "${OFED_C}" > dev/mlx5/mlx5_ib/mlx5_ib_mad.c optional mlx5ib pci > ofed \ > compile-with "${OFED_C}" > dev/mlx5/mlx5_ib/mlx5_ib_main.c optional mlx5ib pci > ofed \ > @@ -4633,10 +4637,10 @@ dev/mlx5/mlx5_ib/mlx5_ib_mr.c optional > mlx5ib pci of compile-with "${OFED_C}" > dev/mlx5/mlx5_ib/mlx5_ib_qp.c optional mlx5ib pci > ofed \ > compile-with "${OFED_C}" > -dev/mlx5/mlx5_ib/mlx5_ib_roce.c optional mlx5ib pci > ofed \ > - compile-with "${OFED_C}" > dev/mlx5/mlx5_ib/mlx5_ib_srq.c optional mlx5ib pci > ofed \ > compile-with "${OFED_C}" > +dev/mlx5/mlx5_ib/mlx5_ib_virt.c optional mlx5ib pci > ofed \ > + compile-with "${OFED_C}" > > dev/mlx5/mlx5_core/mlx5_alloc.c optional mlx5 pci > \ > compile-with "${OFED_C}" > @@ -4694,43 +4698,6 @@ dev/mlx5/mlx5_en/mlx5_en_flow_table.c optional > mlx5en dev/mlx5/mlx5_en/mlx5_en_rx.c optional mlx5en pci inet > inet6 \ compile-with "${OFED_C}" > dev/mlx5/mlx5_en/mlx5_en_txrx.c optional mlx5en pci inet > inet6 \ > - compile-with "${OFED_C}" > - > -ofed/drivers/infiniband/hw/mthca/mthca_allocator.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_av.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_catas.c optional mthca > \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_cmd.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_cq.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_eq.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_mad.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_main.c optional mthca > \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_mcg.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_memfree.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_mr.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_pd.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_profile.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_provider.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_qp.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_reset.c optional mthca > \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_srq.c optional mthca \ > - compile-with "${OFED_C}" > -ofed/drivers/infiniband/hw/mthca/mthca_uar.c optional mthca \ > compile-with "${OFED_C}" > > # crypto support > > Modified: head/sys/conf/kern.pre.mk > ============================================================================== > --- head/sys/conf/kern.pre.mk Fri Nov 24 14:29:32 2017 (r326168) > +++ head/sys/conf/kern.pre.mk Fri Nov 24 14:50:28 2017 (r326169) > @@ -178,9 +178,10 @@ LINUXKPI_C= ${NORMAL_C} ${LINUXKPI_INCLUDES} > > # Infiniband C flags. Correct include paths and omit errors that linux > # does not honor. > -OFEDINCLUDES= -I$S/ofed/include ${LINUXKPI_INCLUDES} > +OFEDINCLUDES= -I$S/ofed/include -I$S/ofed/include/uapi > ${LINUXKPI_INCLUDES} > OFEDNOERR= -Wno-cast-qual -Wno-pointer-arith > -OFEDCFLAGS= ${CFLAGS:N-I*} ${OFEDINCLUDES} ${CFLAGS:M-I*} ${OFEDNOERR} > +OFEDCFLAGS= ${CFLAGS:N-I*} -DCONFIG_INFINIBAND_USER_MEM \ > + ${OFEDINCLUDES} ${CFLAGS:M-I*} ${OFEDNOERR} > OFED_C_NOIMP= ${CC} -c -o ${.TARGET} ${OFEDCFLAGS} ${WERROR} ${PROF} > OFED_C= ${OFED_C_NOIMP} ${.IMPSRC} > > > Modified: head/sys/contrib/rdma/krping/krping.c > ============================================================================== > --- head/sys/contrib/rdma/krping/krping.c Fri Nov 24 14:29:32 2017 > (r326168) +++ head/sys/contrib/rdma/krping/krping.c Fri Nov 24 14:50:28 > 2017 (r326169) @@ -54,13 +54,14 @@ __FBSDID("$FreeBSD$"); > #include "krping.h" > #include "getopt.h" > > +#define PFX "krping: " > + > extern int krping_debug; > -#define DEBUG_LOG(cb, x...) if (krping_debug) log(LOG_INFO, x) > -#define PRINTF(cb, x...) log(LOG_INFO, x) > +#define DEBUG_LOG(...) do { if (krping_debug) log(LOG_INFO, __VA_ARGS__); } > while (0) > #define BIND_INFO 1 > > MODULE_AUTHOR("Steve Wise"); > -MODULE_DESCRIPTION("RDMA ping client/server"); > +MODULE_DESCRIPTION("RDMA ping server"); > MODULE_LICENSE("Dual BSD/GPL"); > MODULE_VERSION(krping, 1); > MODULE_DEPEND(krping, linuxkpi, 1, 1, 1); > @@ -77,9 +78,7 @@ typedef uint64_t cycles_t; > > enum mem_type { > DMA = 1, > - FASTREG = 2, > - MW = 3, > - MR = 4 > + REG = 2, > }; > > static const struct krping_option krping_opts[] = { > @@ -92,7 +91,6 @@ static const struct krping_option krping_opts[] = { > {"validate", OPT_NOPARAM, 'V'}, > {"server", OPT_NOPARAM, 's'}, > {"client", OPT_NOPARAM, 'c'}, > - {"mem_mode", OPT_STRING, 'm'}, > {"server_inv", OPT_NOPARAM, 'I'}, > {"wlat", OPT_NOPARAM, 'l'}, > {"rlat", OPT_NOPARAM, 'L'}, > @@ -102,14 +100,14 @@ static const struct krping_option krping_opts[] = { > {"poll", OPT_NOPARAM, 'P'}, > {"local_dma_lkey", OPT_NOPARAM, 'Z'}, > {"read_inv", OPT_NOPARAM, 'R'}, > - {"fr", OPT_INT, 'f'}, > + {"fr", OPT_NOPARAM, 'f'}, > {NULL, 0, 0} > }; > > #define htonll(x) cpu_to_be64((x)) > #define ntohll(x) cpu_to_be64((x)) > > -static struct mutex krping_mutex; > +static DEFINE_MUTEX(krping_mutex); > > /* > * List of running krping threads. > @@ -117,6 +115,13 @@ static struct mutex krping_mutex; > static LIST_HEAD(krping_cbs); > > /* > + * Invoke like this, one on each side, using the server's address on > + * the RDMA device (iw%d): > + * > + * /bin/echo server,port=9999,addr=192.168.69.142,validate > /proc/krping > + * /bin/echo client,port=9999,addr=192.168.69.142,validate > /proc/krping > + * /bin/echo client,port=9999,addr6=2001:db8:0:f101::1,validate > > /proc/krping > + * > * krping "ping/pong" loop: > * client sends source rkey/addr/len > * server receives source rkey/add/len > @@ -165,42 +170,35 @@ struct krping_rdma_info { > * Control block struct. > */ > struct krping_cb { > - void *cookie; > int server; /* 0 iff client */ > struct ib_cq *cq; > struct ib_pd *pd; > struct ib_qp *qp; > > - enum mem_type mem; > struct ib_mr *dma_mr; > > struct ib_fast_reg_page_list *page_list; > int page_list_len; > - struct ib_send_wr fastreg_wr; > + struct ib_reg_wr reg_mr_wr; > struct ib_send_wr invalidate_wr; > - struct ib_mr *fastreg_mr; > + struct ib_mr *reg_mr; > int server_invalidate; > int read_inv; > u8 key; > > - struct ib_mw *mw; > - struct ib_mw_bind bind_attr; > - > struct ib_recv_wr rq_wr; /* recv work request record */ > struct ib_sge recv_sgl; /* recv single SGE */ > - struct krping_rdma_info recv_buf;/* malloc'd buffer */ > + struct krping_rdma_info recv_buf __aligned(16); /* malloc'd buffer */ > u64 recv_dma_addr; > DECLARE_PCI_UNMAP_ADDR(recv_mapping) > - struct ib_mr *recv_mr; > > struct ib_send_wr sq_wr; /* send work requrest record */ > struct ib_sge send_sgl; > - struct krping_rdma_info send_buf;/* single send buf */ > + struct krping_rdma_info send_buf __aligned(16); /* single send buf */ > u64 send_dma_addr; > DECLARE_PCI_UNMAP_ADDR(send_mapping) > - struct ib_mr *send_mr; > > - struct ib_send_wr rdma_sq_wr; /* rdma work request record */ > + struct ib_rdma_wr rdma_sq_wr; /* rdma work request record */ > struct ib_sge rdma_sgl; /* rdma single SGE */ > char *rdma_buf; /* used as rdma sink */ > u64 rdma_dma_addr; > @@ -221,12 +219,9 @@ struct krping_cb { > struct krping_stats stats; > > uint16_t port; /* dst port in NBO */ > - union { > - struct in_addr v4; > - struct in6_addr v6; > - } addr; /* dst addr in NBO */ > - int addr_type; /* AF_INET or AF_INET6 */ > + u8 addr[16] __aligned(8); /* dst addr in NBO */ > char *addr_str; /* dst addr string */ > + uint8_t addr_type; /* ADDR_FAMILY - IPv4/V6 */ > int verbose; /* verbose logging */ > int count; /* ping count */ > int size; /* ping data size */ > @@ -238,8 +233,7 @@ struct krping_cb { > int poll; /* poll or block for rlat test */ > int txdepth; /* SQ depth */ > int local_dma_lkey; /* use 0 for lkey */ > - int frtest; /* fastreg test */ > - int testnum; > + int frtest; /* reg test */ > > /* CM stuff */ > struct rdma_cm_id *cm_id; /* connection on client side,*/ > @@ -254,39 +248,34 @@ static int krping_cma_event_handler(struct rdma_cm_id > int ret; > struct krping_cb *cb = cma_id->context; > > - DEBUG_LOG(cb, "cma_event type %d cma_id %p (%s)\n", event->event, > - cma_id, (cma_id == cb->cm_id) ? "parent" : "child"); > + DEBUG_LOG("cma_event type %d cma_id %p (%s)\n", event->event, cma_id, > + (cma_id == cb->cm_id) ? "parent" : "child"); > > switch (event->event) { > case RDMA_CM_EVENT_ADDR_RESOLVED: > cb->state = ADDR_RESOLVED; > ret = rdma_resolve_route(cma_id, 2000); > if (ret) { > - PRINTF(cb, "rdma_resolve_route error %d\n", ret); > + printk(KERN_ERR PFX "rdma_resolve_route error %d\n", > + ret); > wake_up_interruptible(&cb->sem); > } > break; > > case RDMA_CM_EVENT_ROUTE_RESOLVED: > cb->state = ROUTE_RESOLVED; > - cb->child_cm_id = cma_id; > wake_up_interruptible(&cb->sem); > break; > > case RDMA_CM_EVENT_CONNECT_REQUEST: > - if (cb->state == IDLE) { > - cb->state = CONNECT_REQUEST; > - cb->child_cm_id = cma_id; > - } else { > - PRINTF(cb, "Received connection request in wrong state" > - " (%d)\n", cb->state); > - } > - DEBUG_LOG(cb, "child cma %p\n", cb->child_cm_id); > + cb->state = CONNECT_REQUEST; > + cb->child_cm_id = cma_id; > + DEBUG_LOG("child cma %p\n", cb->child_cm_id); > wake_up_interruptible(&cb->sem); > break; > > case RDMA_CM_EVENT_ESTABLISHED: > - DEBUG_LOG(cb, "ESTABLISHED\n"); > + DEBUG_LOG("ESTABLISHED\n"); > if (!cb->server) { > cb->state = CONNECTED; > } > @@ -298,24 +287,24 @@ static int krping_cma_event_handler(struct rdma_cm_id > case RDMA_CM_EVENT_CONNECT_ERROR: > case RDMA_CM_EVENT_UNREACHABLE: > case RDMA_CM_EVENT_REJECTED: > - PRINTF(cb, "cma event %d, error %d\n", event->event, > + printk(KERN_ERR PFX "cma event %d, error %d\n", event->event, > event->status); > cb->state = ERROR; > wake_up_interruptible(&cb->sem); > break; > > case RDMA_CM_EVENT_DISCONNECTED: > - PRINTF(cb, "DISCONNECT EVENT...\n"); > + printk(KERN_ERR PFX "DISCONNECT EVENT...\n"); > cb->state = ERROR; > wake_up_interruptible(&cb->sem); > break; > > case RDMA_CM_EVENT_DEVICE_REMOVAL: > - PRINTF(cb, "cma detected device removal!!!!\n"); > + printk(KERN_ERR PFX "cma detected device removal!!!!\n"); > break; > > default: > - PRINTF(cb, "oof bad type!\n"); > + printk(KERN_ERR PFX "oof bad type!\n"); > wake_up_interruptible(&cb->sem); > break; > } > @@ -325,7 +314,7 @@ static int krping_cma_event_handler(struct rdma_cm_id > static int server_recv(struct krping_cb *cb, struct ib_wc *wc) > { > if (wc->byte_len != sizeof(cb->recv_buf)) { > - PRINTF(cb, "Received bogus data, size %d\n", > + printk(KERN_ERR PFX "Received bogus data, size %d\n", > wc->byte_len); > return -1; > } > @@ -333,7 +322,7 @@ static int server_recv(struct krping_cb *cb, struct ib > cb->remote_rkey = ntohl(cb->recv_buf.rkey); > cb->remote_addr = ntohll(cb->recv_buf.buf); > cb->remote_len = ntohl(cb->recv_buf.size); > - DEBUG_LOG(cb, "Received rkey %x addr %llx len %d from peer\n", > + DEBUG_LOG("Received rkey %x addr %llx len %d from peer\n", > cb->remote_rkey, (unsigned long long)cb->remote_addr, > cb->remote_len); > > @@ -348,7 +337,7 @@ static int server_recv(struct krping_cb *cb, struct ib > static int client_recv(struct krping_cb *cb, struct ib_wc *wc) > { > if (wc->byte_len != sizeof(cb->recv_buf)) { > - PRINTF(cb, "Received bogus data, size %d\n", > + printk(KERN_ERR PFX "Received bogus data, size %d\n", > wc->byte_len); > return -1; > } > @@ -370,18 +359,22 @@ static void krping_cq_event_handler(struct ib_cq *cq, > > BUG_ON(cb->cq != cq); > if (cb->state == ERROR) { > - PRINTF(cb, "cq completion in ERROR state\n"); > + printk(KERN_ERR PFX "cq completion in ERROR state\n"); > return; > } > - if (!cb->wlat && !cb->rlat && !cb->bw && !cb->frtest) > + if (cb->frtest) { > + printk(KERN_ERR PFX "cq completion event in frtest!\n"); > + return; > + } > + if (!cb->wlat && !cb->rlat && !cb->bw) > ib_req_notify_cq(cb->cq, IB_CQ_NEXT_COMP); > while ((ret = ib_poll_cq(cb->cq, 1, &wc)) == 1) { > if (wc.status) { > if (wc.status == IB_WC_WR_FLUSH_ERR) { > - DEBUG_LOG(cb, "cq flushed\n"); > + DEBUG_LOG("cq flushed\n"); > continue; > } else { > - PRINTF(cb, "cq completion failed with " > + printk(KERN_ERR PFX "cq completion failed with " > "wr_id %jx status %d opcode %d > vender_err %x\n", > (uintmax_t)wc.wr_id, wc.status, > wc.opcode, > wc.vendor_err); goto error; > @@ -390,44 +383,44 @@ static void krping_cq_event_handler(struct ib_cq *cq, > > switch (wc.opcode) { > case IB_WC_SEND: > - DEBUG_LOG(cb, "send completion\n"); > + DEBUG_LOG("send completion\n"); > cb->stats.send_bytes += cb->send_sgl.length; > cb->stats.send_msgs++; > break; > > case IB_WC_RDMA_WRITE: > - DEBUG_LOG(cb, "rdma write completion\n"); > - cb->stats.write_bytes += cb->rdma_sq_wr.sg_list->length; > + DEBUG_LOG("rdma write completion\n"); > + cb->stats.write_bytes += > cb->rdma_sq_wr.wr.sg_list->length; > cb->stats.write_msgs++; > cb->state = RDMA_WRITE_COMPLETE; > wake_up_interruptible(&cb->sem); > break; > > case IB_WC_RDMA_READ: > - DEBUG_LOG(cb, "rdma read completion\n"); > - cb->stats.read_bytes += cb->rdma_sq_wr.sg_list->length; > + DEBUG_LOG("rdma read completion\n"); > + cb->stats.read_bytes += > cb->rdma_sq_wr.wr.sg_list->length; > cb->stats.read_msgs++; > cb->state = RDMA_READ_COMPLETE; > wake_up_interruptible(&cb->sem); > break; > > case IB_WC_RECV: > - DEBUG_LOG(cb, "recv completion\n"); > + DEBUG_LOG("recv completion\n"); > cb->stats.recv_bytes += sizeof(cb->recv_buf); > cb->stats.recv_msgs++; > - if (cb->wlat || cb->rlat || cb->bw || cb->frtest) > + if (cb->wlat || cb->rlat || cb->bw) > ret = server_recv(cb, &wc); > else > ret = cb->server ? server_recv(cb, &wc) : > client_recv(cb, &wc); > if (ret) { > - PRINTF(cb, "recv wc error: %d\n", ret); > + printk(KERN_ERR PFX "recv wc error: %d\n", ret); > goto error; > } > > ret = ib_post_recv(cb->qp, &cb->rq_wr, &bad_wr); > if (ret) { > - PRINTF(cb, "post recv error: %d\n", > + printk(KERN_ERR PFX "post recv error: %d\n", > ret); > goto error; > } > @@ -435,14 +428,14 @@ static void krping_cq_event_handler(struct ib_cq *cq, > break; > > default: > - PRINTF(cb, > + printk(KERN_ERR PFX > "%s:%d Unexpected opcode %d, Shutting down\n", > __func__, __LINE__, wc.opcode); > goto error; > } > } > if (ret) { > - PRINTF(cb, "poll error %d\n", ret); > + printk(KERN_ERR PFX "poll error %d\n", ret); > goto error; > } > return; > @@ -456,7 +449,7 @@ static int krping_accept(struct krping_cb *cb) > struct rdma_conn_param conn_param; > int ret; > > - DEBUG_LOG(cb, "accepting client connection request\n"); > + DEBUG_LOG("accepting client connection request\n"); > > memset(&conn_param, 0, sizeof conn_param); > conn_param.responder_resources = 1; > @@ -464,14 +457,14 @@ static int krping_accept(struct krping_cb *cb) > > ret = rdma_accept(cb->child_cm_id, &conn_param); > if (ret) { > - PRINTF(cb, "rdma_accept error: %d\n", ret); > + printk(KERN_ERR PFX "rdma_accept error: %d\n", ret); > return ret; > } > > - if (!cb->wlat && !cb->rlat && !cb->bw && !cb->frtest) { > + if (!cb->wlat && !cb->rlat && !cb->bw) { > wait_event_interruptible(cb->sem, cb->state >= CONNECTED); > if (cb->state == ERROR) { > - PRINTF(cb, "wait for CONNECTED state %d\n", > + printk(KERN_ERR PFX "wait for CONNECTED state %d\n", > cb->state); > return -1; > } > @@ -483,278 +476,120 @@ static void krping_setup_wr(struct krping_cb *cb) > { > cb->recv_sgl.addr = cb->recv_dma_addr; > cb->recv_sgl.length = sizeof cb->recv_buf; > - if (cb->local_dma_lkey) > - cb->recv_sgl.lkey = cb->qp->device->local_dma_lkey; > - else if (cb->mem == DMA) > - cb->recv_sgl.lkey = cb->dma_mr->lkey; > - else > - cb->recv_sgl.lkey = cb->recv_mr->lkey; > + cb->recv_sgl.lkey = cb->pd->local_dma_lkey; > cb->rq_wr.sg_list = &cb->recv_sgl; > cb->rq_wr.num_sge = 1; > > cb->send_sgl.addr = cb->send_dma_addr; > cb->send_sgl.length = sizeof cb->send_buf; > - if (cb->local_dma_lkey) > - cb->send_sgl.lkey = cb->qp->device->local_dma_lkey; > - else if (cb->mem == DMA) > - cb->send_sgl.lkey = cb->dma_mr->lkey; > - else > - cb->send_sgl.lkey = cb->send_mr->lkey; > + cb->send_sgl.lkey = cb->pd->local_dma_lkey; > > cb->sq_wr.opcode = IB_WR_SEND; > cb->sq_wr.send_flags = IB_SEND_SIGNALED; > cb->sq_wr.sg_list = &cb->send_sgl; > cb->sq_wr.num_sge = 1; > > - if (cb->server || cb->wlat || cb->rlat || cb->bw || cb->frtest) { > + if (cb->server || cb->wlat || cb->rlat || cb->bw) { > cb->rdma_sgl.addr = cb->rdma_dma_addr; > - if (cb->mem == MR) > - cb->rdma_sgl.lkey = cb->rdma_mr->lkey; > - cb->rdma_sq_wr.send_flags = IB_SEND_SIGNALED; > - cb->rdma_sq_wr.sg_list = &cb->rdma_sgl; > - cb->rdma_sq_wr.num_sge = 1; > + cb->rdma_sq_wr.wr.send_flags = IB_SEND_SIGNALED; > + cb->rdma_sq_wr.wr.sg_list = &cb->rdma_sgl; > + cb->rdma_sq_wr.wr.num_sge = 1; > } > > - switch(cb->mem) { > - case FASTREG: > + /* > + * A chain of 2 WRs, INVALDATE_MR + REG_MR. > + * both unsignaled. The client uses them to reregister > + * the rdma buffers with a new key each iteration. > + */ > + cb->reg_mr_wr.wr.opcode = IB_WR_REG_MR; > + cb->reg_mr_wr.mr = cb->reg_mr; > > - /* > - * A chain of 2 WRs, INVALDATE_MR + FAST_REG_MR. > - * both unsignaled. The client uses them to reregister > - * the rdma buffers with a new key each iteration. > - */ > - cb->fastreg_wr.opcode = IB_WR_FAST_REG_MR; > - cb->fastreg_wr.wr.fast_reg.page_shift = PAGE_SHIFT; > - cb->fastreg_wr.wr.fast_reg.length = cb->size; > - cb->fastreg_wr.wr.fast_reg.page_list = cb->page_list; > - cb->fastreg_wr.wr.fast_reg.page_list_len = cb->page_list_len; > - > - cb->invalidate_wr.next = &cb->fastreg_wr; > - cb->invalidate_wr.opcode = IB_WR_LOCAL_INV; > - break; > - case MW: > - cb->bind_attr.wr_id = 0xabbaabba; > - cb->bind_attr.send_flags = 0; /* unsignaled */ > -#ifdef BIND_INFO > - cb->bind_attr.bind_info.length = cb->size; > -#else > - cb->bind_attr.length = cb->size; > -#endif > - break; > - default: > - break; > - } > + cb->invalidate_wr.next = &cb->reg_mr_wr.wr; > + cb->invalidate_wr.opcode = IB_WR_LOCAL_INV; > } > > static int krping_setup_buffers(struct krping_cb *cb) > { > int ret; > - struct ib_phys_buf buf; > - u64 iovbase; > > - DEBUG_LOG(cb, "krping_setup_buffers called on cb %p\n", cb); > + DEBUG_LOG(PFX "krping_setup_buffers called on cb %p\n", cb); > > - cb->recv_dma_addr = ib_dma_map_single(cb->pd->device, > + cb->recv_dma_addr = ib_dma_map_single(cb->pd->device, > &cb->recv_buf, > sizeof(cb->recv_buf), DMA_BIDIRECTIONAL); > pci_unmap_addr_set(cb, recv_mapping, cb->recv_dma_addr); > - cb->send_dma_addr = ib_dma_map_single(cb->pd->device, > + cb->send_dma_addr = ib_dma_map_single(cb->pd->device, > &cb->send_buf, sizeof(cb->send_buf), > DMA_BIDIRECTIONAL); > pci_unmap_addr_set(cb, send_mapping, cb->send_dma_addr); > > - if (cb->mem == DMA) { > - cb->dma_mr = ib_get_dma_mr(cb->pd, IB_ACCESS_LOCAL_WRITE| > - IB_ACCESS_REMOTE_READ| > - IB_ACCESS_REMOTE_WRITE); > - if (IS_ERR(cb->dma_mr)) { > - DEBUG_LOG(cb, "reg_dmamr failed\n"); > - ret = PTR_ERR(cb->dma_mr); > - goto bail; > - } > - } else { > - if (!cb->local_dma_lkey) { > - buf.addr = cb->recv_dma_addr; > - buf.size = sizeof cb->recv_buf; > - DEBUG_LOG(cb, "recv buf dma_addr %jx size %d\n", > - (uintmax_t)buf.addr, (int)buf.size); > - iovbase = cb->recv_dma_addr; > - cb->recv_mr = ib_reg_phys_mr(cb->pd, &buf, 1, > - IB_ACCESS_LOCAL_WRITE, > - &iovbase); > - > - if (IS_ERR(cb->recv_mr)) { > - DEBUG_LOG(cb, "recv_buf reg_mr failed\n"); > - ret = PTR_ERR(cb->recv_mr); > - goto bail; > - } > - > - buf.addr = cb->send_dma_addr; > - buf.size = sizeof cb->send_buf; > - DEBUG_LOG(cb, "send buf dma_addr %jx size %d\n", > - (uintmax_t)buf.addr, (int)buf.size); > - iovbase = cb->send_dma_addr; > - cb->send_mr = ib_reg_phys_mr(cb->pd, &buf, 1, > - 0, &iovbase); > - > - if (IS_ERR(cb->send_mr)) { > - DEBUG_LOG(cb, "send_buf reg_mr failed\n"); > - ret = PTR_ERR(cb->send_mr); > - goto bail; > - } > - } > - } > - > - cb->rdma_buf = kmalloc(cb->size, GFP_KERNEL); > + cb->rdma_buf = ib_dma_alloc_coherent(cb->pd->device, cb->size, > + &cb->rdma_dma_addr, > + GFP_KERNEL); > if (!cb->rdma_buf) { > - DEBUG_LOG(cb, "rdma_buf malloc failed\n"); > + DEBUG_LOG(PFX "rdma_buf allocation failed\n"); > ret = -ENOMEM; > goto bail; > } > - > - cb->rdma_dma_addr = ib_dma_map_single(cb->pd->device, > - cb->rdma_buf, cb->size, > - DMA_BIDIRECTIONAL); > pci_unmap_addr_set(cb, rdma_mapping, cb->rdma_dma_addr); > - if (cb->mem != DMA) { > - switch (cb->mem) { > - case FASTREG: > - cb->page_list_len = (((cb->size - 1) & PAGE_MASK) + > - PAGE_SIZE) >> PAGE_SHIFT; > - cb->page_list = ib_alloc_fast_reg_page_list( > - cb->pd->device, > - cb->page_list_len); > - if (IS_ERR(cb->page_list)) { > - DEBUG_LOG(cb, "recv_buf reg_mr failed\n"); > - ret = PTR_ERR(cb->page_list); > - goto bail; > - } > - cb->fastreg_mr = ib_alloc_fast_reg_mr(cb->pd, > - cb->page_list->max_page_list_len); > - if (IS_ERR(cb->fastreg_mr)) { > - DEBUG_LOG(cb, "recv_buf reg_mr failed\n"); > - ret = PTR_ERR(cb->fastreg_mr); > - goto bail; > - } > - DEBUG_LOG(cb, "fastreg rkey 0x%x page_list %p" > - " page_list_len %u\n", cb->fastreg_mr->rkey, > - cb->page_list, cb->page_list_len); > - break; > - case MW: > - cb->mw = ib_alloc_mw(cb->pd,IB_MW_TYPE_1); > - if (IS_ERR(cb->mw)) { > - DEBUG_LOG(cb, "recv_buf alloc_mw failed\n"); > - ret = PTR_ERR(cb->mw); > - goto bail; > - } > - DEBUG_LOG(cb, "mw rkey 0x%x\n", cb->mw->rkey); > - /*FALLTHROUGH*/ > - case MR: > - buf.addr = cb->rdma_dma_addr; > - buf.size = cb->size; > - iovbase = cb->rdma_dma_addr; > - cb->rdma_mr = ib_reg_phys_mr(cb->pd, &buf, 1, > - IB_ACCESS_LOCAL_WRITE| > - IB_ACCESS_REMOTE_READ| > - IB_ACCESS_REMOTE_WRITE, > - &iovbase); > - if (IS_ERR(cb->rdma_mr)) { > - DEBUG_LOG(cb, "rdma_buf reg_mr failed\n"); > - ret = PTR_ERR(cb->rdma_mr); > - goto bail; > - } > - DEBUG_LOG(cb, "rdma buf dma_addr %jx size %d mr rkey > 0x%x\n", > - (uintmax_t)buf.addr, (int)buf.size, > cb->rdma_mr->rkey); > - break; > - default: > - ret = -EINVAL; > - goto bail; > - break; > - } > + cb->page_list_len = (((cb->size - 1) & PAGE_MASK) + PAGE_SIZE) > + >> PAGE_SHIFT; > + cb->reg_mr = ib_alloc_mr(cb->pd, IB_MR_TYPE_MEM_REG, > + cb->page_list_len); > + if (IS_ERR(cb->reg_mr)) { > + ret = PTR_ERR(cb->reg_mr); > + DEBUG_LOG(PFX "recv_buf reg_mr failed %d\n", ret); > + goto bail; > } > + DEBUG_LOG(PFX "reg rkey 0x%x page_list_len %u\n", > + cb->reg_mr->rkey, cb->page_list_len); > > - if (!cb->server || cb->wlat || cb->rlat || cb->bw || cb->frtest) { > + if (!cb->server || cb->wlat || cb->rlat || cb->bw) { > > - cb->start_buf = kmalloc(cb->size, GFP_KERNEL); > + cb->start_buf = ib_dma_alloc_coherent(cb->pd->device, cb->size, > + &cb->start_dma_addr, > + GFP_KERNEL); > if (!cb->start_buf) { > - DEBUG_LOG(cb, "start_buf malloc failed\n"); > + DEBUG_LOG(PFX "start_buf malloc failed\n"); > ret = -ENOMEM; > goto bail; > } > - > - cb->start_dma_addr = ib_dma_map_single(cb->pd->device, > - cb->start_buf, cb->size, > - DMA_BIDIRECTIONAL); > pci_unmap_addr_set(cb, start_mapping, cb->start_dma_addr); > - > - if (cb->mem == MR || cb->mem == MW) { > - unsigned flags = IB_ACCESS_REMOTE_READ; > - > - if (cb->wlat || cb->rlat || cb->bw || cb->frtest) { > - flags |= IB_ACCESS_LOCAL_WRITE | > - IB_ACCESS_REMOTE_WRITE; > - } > - > - buf.addr = cb->start_dma_addr; > > *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** > _______________________________________________ > svn-src-head@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-head > To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org" Buildworld failure: Building /usr/obj/usr/src/amd64.amd64/contrib/ofed/opensm/complib/cl_pool.o --- cl_nodenamemap.o --- /usr/src/contrib/ofed/opensm/complib/cl_nodenamemap.c:77:11: error: conflicting types for 'open_node_name_map' nn_map_t *open_node_name_map(const char *node_name_map) ^ /usr/include/infiniband/complib/cl_nodenamemap.h:57:11: note: previous declaration is here nn_map_t *open_node_name_map(char *node_name_map); ^ 1 error generated. *** [cl_nodenamemap.o] Error code 1 -- O. Hartmann Ich widerspreche der Nutzung oder Übermittlung meiner Daten für Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).
pgpgK5w7Ms3HY.pgp
Description: OpenPGP digital signature
