Hi Julius, You don’t need any patch for Ubuntu it should work out of the box on 21.01… This patch is CentOS specific because the default libdir on CentOS is ‘lib64' (while it is ‘lib' on Ubuntu/Debian). On Ubuntu, you only need to edit dpdk.mk to enable MLX4_PMD, MLX5_PMD and MLX5_COMMON_PMD. Internally it works like this: - rdma-core (and the libibverbs *static* library) is built and installed in some internal staging directory (build-root/install-vpp-native/external/lib/). - DPDK is configured to be statically linked against this libibverb statis library, and is also statically built. - then the VPP dpdk plugin is statically linked against DPDK.
The patch fixes the first step, as on CentOS, cmake will install the rdma-core libraries in build-root/install-vpp-native/external/lib64 while the DPDK build system will look for them in build-root/install-vpp-native/external/lib. Note that this patch is not “specific” to CentOS, but is *required* for CentOS support (and is merged in master). Best regards, Mohammed > On 16 Apr 2021, at 16:55, Július Milan <jmilan....@gmail.com> wrote: > > Hi Mohammed > > I am trying to create the patch similar to yours: > https://gerrit.fd.io/r/c/vpp/+/31876 <https://gerrit.fd.io/r/c/vpp/+/31876> > But being able to run on Ubuntu. > Can you point me a bit? I would like to understand how your patch is CentOS > specific. > Probably it has something to do with linking of libibverbs.so into > dpdk_plugin, which I was not able to achieve yet. > Or am I completely wrong? > > Best regards > Julius > > On Fri, 16 Apr 2021 at 09:49, Mohammed Hawari <moham...@hawari.fr > <mailto:moham...@hawari.fr>> wrote: > > Hi Chetan, > >> On 16 Apr 2021, at 07:04, chetan bhasin <chetan.bhasin...@gmail.com >> <mailto:chetan.bhasin...@gmail.com>> wrote: >> >> Hi Mohammed , >> >> After applying the patch you have mentioned , I am no longer facing the >> compilation issue. >> >> Thanks for your support. >> >> With Static RDMA linking , do we need to do anything extra while bringing >> our application on the server having Mellanox Nic ? > > No we don’t, it should work out of the box! >> >> Thanks & Regards, >> Chetan > > Regards > > Mohammed >> >> On Wed, Apr 14, 2021 at 8:33 PM chetan bhasin via lists.fd.io >> <http://lists.fd.io/> <chetan.bhasin017=gmail....@lists.fd.io >> <mailto:gmail....@lists.fd.io>> wrote: >> Thanks a lot Mohammed! >> >> Let me try with the rdma-core and patch mentioned by you and >> >> Thanks, >> Chetan >> >> On Wed, Apr 14, 2021 at 6:20 PM Mohammed Hawari <moham...@hawari.fr >> <mailto:moham...@hawari.fr>> wrote: >> Hi Chetan >> >> You are building DPDK with Mellanox support relying on dlopen and the glue >> library which is fine. The default approach implemented in the VPP build >> system consists of statically linking DPDK with the the libraries provided >> by rdma-core (as it’s also built by the VPP build system). The default >> approach should work fine, provided this patch >> https://gerrit.fd.io/r/c/vpp/+/31876 <https://gerrit.fd.io/r/c/vpp/+/31876> >> is applied (specific to CentOS). But what you should work just fine. Now the >> reason why you need to comment dpdk_depends is kind of mysterious here… >> >> Regards, >> >> Mohammed >>> On 14 Apr 2021, at 10:07, chetan bhasin <chetan.bhasin...@gmail.com >>> <mailto:chetan.bhasin...@gmail.com>> wrote: >>> >>> Hi, >>> >>> I have to do the following to enable Mellanox compilation under VPP21.01 >>> with dpdk 20.11. >>> >>> If I dont comment rdma-core dependencies , it will lead to undefined >>> symbols . >>> >>> Can anybody please correct , is this the right way to do it? >>> >>> -DPDK_MLX4_PMD ?= n >>> -DPDK_MLX5_PMD ?= n >>> -DPDK_MLX5_COMMON_PMD ?= n >>> +DPDK_MLX4_PMD := y >>> +DPDK_MLX5_PMD := y >>> +DPDK_MLX5_COMMON_PMD = y >>> >>> -DPDK_MLX_IBV_LINK ?= static >>> +DPDK_MLX_IBV_LINK := dlopen >>> >>> -dpdk_depends := rdma-core $(if $(ARCH_X86_64), ipsec-mb) >>> +#dpdk_depends := rdma-core $(if $(ARCH_X86_64), ipsec-mb) >>> >>> Thanks, >>> Chetan >>> >>> On Tue, Apr 6, 2021 at 4:10 PM chetan bhasin via lists.fd.io >>> <http://lists.fd.io/> <chetan.bhasin017=gmail....@lists.fd.io >>> <mailto:gmail....@lists.fd.io>> wrote: >>> Thanks Guys ! >>> >>> We have a requirement to compile vppp with DPDK mellanox driver . >>> >>> Right now issue what i understood is , when we compile vpp_2101 , using >>> make build-release , it builds mellanox , but when it is trying to install >>> vpp-ext-dep rpm , it got failed because glue library created of mellanox >>> has some undefined symbols , >>> For your reference , i have set following in dpdk.mk <http://dpdk.mk/> file >>> >>> -DPDK_MLX4_PMD ?= n >>> -DPDK_MLX5_PMD ?= n >>> +DPDK_MLX4_PMD := y >>> +DPDK_MLX5_PMD := y >>> -DPDK_MLX_IBV_LINK ?= static >>> +DPDK_MLX_IBV_LINK := shared >>> >>> please check below logs for reference- >>> >>> FAILED: drivers/net/mlx4/librte_net_mlx4_glue.so.21.0 >>> cc -o drivers/net/mlx4/librte_net_mlx4_glue.so.21.0 >>> 'drivers/net/mlx4/8672f8e@@rte_net_mlx4_glue@sha/mlx4_glue.c.o' >>> -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group >>> -Wl,-soname,librte_net_mlx4_glue.so.21.0 -Wl,--no-as-needed -pthread -lm >>> -ldl -lnuma -Wl,-export-dynamic -Wl,-h,librte_net_mlx4_glue.so.21.0 >>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libmlx4.so >>> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libibverbs.so >>> -Wl,--end-group >>> drivers/net/mlx4/8672f8e@@rte_net_mlx4_glue@sha/mlx4_glue.c.o: In function >>> `mlx4_glue_reg_mr': >>> mlx4_glue.c:(.text+0x277): undefined reference to `ibv_reg_mr_iova2' >>> collect2: error: ld returned 1 exit status >>> [8/547] Compiling C object >>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_flow.c.o' >>> >>> ninja: build stopped: subcommand failed. >>> >>> On Thu, Apr 1, 2021 at 4:33 PM Юрий Иванов <format_...@outlook.com >>> <mailto:format_...@outlook.com>> wrote: >>> Hi, >>> >>> Or you can use nativa rdma driver as written in this post and build vpp as >>> usual. 😉 >>> >>> Regards. >>> От: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> <vpp-dev@lists.fd.io >>> <mailto:vpp-dev@lists.fd.io>> от имени Mohammed Hawari <moham...@hawari.fr >>> <mailto:moham...@hawari.fr>> >>> Отправлено: 1 апреля 2021 г. 13:16 >>> Кому: chetan bhasin <chetan.bhasin...@gmail.com >>> <mailto:chetan.bhasin...@gmail.com>> >>> Копия: vpp-dev <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>> >>> Тема: Re: [vpp-dev] How to enable Mellanox compilation in VPP 21.01 >>> >>> >>> Hi Chetan, >>> >>> If you are using CentOS, I’d suggest to cherry-pick >>> https://gerrit.fd.io/r/c/vpp/+/31876 >>> <https://gerrit.fd.io/r/c/vpp/+/31876>. Also please change the dpdk.mk >>> <http://dpdk.mk/> to also set DPDK_MLX5_COMMON_PMD = y. I hope this solves >>> your issue. Otherwise, please consider using the rdma native driver that >>> does not rely on DPDK. >>> >>> Regards >>> >>> Mohammed >>>> On 30 Mar 2021, at 12:41, chetan bhasin <chetan.bhasin...@gmail.com >>>> <mailto:chetan.bhasin...@gmail.com>> wrote: >>>> >>>> Hello Everyone, >>>> >>>> I am upgrading to vpp 2101 . I am facing a compilation issue after >>>> enabling Mellanox compilation in dpdk.mk <http://dpdk.mk/> . >>>> >>>> --- a/build/external/packages/dpdk.mk <http://dpdk.mk/> >>>> +++ b/build/external/packages/dpdk.mk <http://dpdk.mk/> >>>> @@ -14,8 +14,8 @@ >>>> DPDK_PKTMBUF_HEADROOM ?= 128 >>>> DPDK_USE_LIBBSD ?= n >>>> DPDK_DEBUG ?= n >>>> -DPDK_MLX4_PMD ?= n >>>> -DPDK_MLX5_PMD ?= n >>>> +DPDK_MLX4_PMD ?= y >>>> +DPDK_MLX5_PMD ?= y >>>> DPDK_MLX5_COMMON_PMD ?= n >>>> >>>> >>>> Getting below errors . Anybody please help here ? >>>> >>>> [1344/1846] Compiling C object >>>> 'drivers/a715181@@tmp_rte_net_nfp@sta/net_nfp_nfpcore_nfp_nsp_cmds.c.o' >>>> [1345/1846] Linking target drivers/librte_net_mlx4.so.21.0 >>>> FAILED: drivers/librte_net_mlx4.so.21.0 >>>> cc -o drivers/librte_net_mlx4.so.21.0 >>>> 'drivers/a715181@@rte_net_mlx4@sha/meson-generated_.._rte_net_mlx4.pmd.c.o' >>>> 'drivers/a715181@@tmp_r >>>> te_net_mlx4@sta/net_mlx4_mlx4.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_ethdev.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta >>>> /net_mlx4_mlx4_flow.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_intr.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_ml >>>> x4_mp.c.o' 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_mr.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_rxq.c.o' 'dri >>>> vers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_rxtx.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_txq.c.o' >>>> 'drivers/a715181@ >>>> @tmp_rte_net_mlx4@sta/net_mlx4_mlx4_utils.c.o' >>>> 'drivers/a715181@@tmp_rte_net_mlx4@sta/net_mlx4_mlx4_glue.c.o' >>>> -Wl,--as-needed -Wl,--no-und >>>> efined -Wl,-O1 -shared -fPIC -Wl,--start-group >>>> -Wl,-soname,librte_net_mlx4.so.21 -Wl,--no-as-needed -pthread -lm -ldl >>>> -lnuma lib/librte_et >>>> hdev.so.21.0 lib/librte_eal.so.21.0 lib/librte_kvargs.so.21.0 >>>> lib/librte_telemetry.so.21.0 lib/librte_net.so.21.0 lib/librte_mbuf.so.21.0 >>>> lib/librte_mempool.so.21.0 lib/librte_ring.so.21.0 >>>> lib/librte_meter.so.21.0 drivers/librte_bus_pci.so.21.0 >>>> lib/librte_pci.so.21.0 drivers/ >>>> librte_bus_vdev.so.21.0 >>>> -Wl,--version-script=/root/tmp/vpp/build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/version.map >>>> -lpt >>>> hread -L/root/tmp/vpp/build-root/install-vpp-native/external/lib64 >>>> -l:libbnxt_re-rdmav25.a -l:libcxgb4-rdmav25.a -l:libefa.a -l:libhns-rdm >>>> av25.a -l:libi40iw-rdmav25.a -l:libmlx4.a -l:libmlx5.a >>>> -l:libmthca-rdmav25.a -l:libocrdma-rdmav25.a -l:libqedr-rdmav25.a >>>> -l:libvmw_pvrdma-rdmav25.a -l:libhfi1verbs-rdmav25.a >>>> -l:libipathverbs-rdmav25.a -l:librxe-rdmav25.a -l:libsiw-rdmav25.a >>>> -l:libibverbs.a -l:librdma_util.a -l:libccan.a -Wl,--end-group >>>> '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' >>>> -Wl,-rpath-link,/root/tmp/vpp/build-root/build-vpp-native/external/build-dpdk/lib >>>> >>>> -Wl,-rpath-link,/root/tmp/vpp/build-root/build-vpp-native/external/build-dpdk/drivers >>>> /bin/ld: cannot find -l:libbnxt_re-rdmav25.a >>>> /bin/ld: cannot find -l:libcxgb4-rdmav25.a >>>> /bin/ld: cannot find -l:libefa.a >>>> /bin/ld: cannot find -l:libhns-rdmav25.a >>>> /bin/ld: cannot find -l:libi40iw-rdmav25.a >>>> /bin/ld: cannot find -l:libmlx4.a >>>> /bin/ld: cannot find -l:libmlx5.a >>>> /bin/ld: cannot find -l:libmthca-rdmav25.a >>>> /bin/ld: cannot find -l:libocrdma-rdmav25.a >>>> /bin/ld: cannot find -l:libqedr-rdmav25.a >>>> /bin/ld: cannot find -l:libvmw_pvrdma-rdmav25.a >>>> /bin/ld: cannot find -l:libhfi1verbs-rdmav25.a >>>> /bin/ld: cannot find -l:libipathverbs-rdmav25.a >>>> /bin/ld: cannot find -l:librxe-rdmav25.a >>>> /bin/ld: cannot find -l:libsiw-rdmav25.a >>>> /bin/ld: cannot find -l:libibverbs.a >>>> /bin/ld: cannot find -l:librdma_util.a >>>> /bin/ld: cannot find -l:libccan.a >>>> collect2: error: ld returned 1 exit status >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >>> >>> >> >> >> >> >> >> >> > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19230): https://lists.fd.io/g/vpp-dev/message/19230 Mute This Topic: https://lists.fd.io/mt/81718770/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
