Lee, Could you check that this patch fixes your issue: https://gerrit.fd.io/r/c/18741/ You may have to use LD_LIBRARY_PATH so DPDK can find librte_pmd_mlx5_glue.so.19.02.0 at runtime.
ben > -----Original Message----- > From: Benoit Ganne (bganne) > Sent: lundi 8 avril 2019 10:02 > To: 'Lee Roberts' <lee.robe...@hpe.com>; vpp-dev@lists.fd.io > Cc: 'Dave Wallace' <dwallac...@gmail.com> > Subject: RE: runtime undefined symbol ibv_cmd_query_port when using VPP > external rdma-core > > Dave, this is probably an issue we want to fix for VPP 19.04. > > Right, there is a conflict between internal VPP rdma-core and external > (typically distribution package) rdma-core when enabling MLX DPDK support. > We compile a very small subset of RDMA core statically to enable VPP RDMA > driver. This subset does not include all the symbols needed by MLX DPDK > driver, hence the issue you ran into. > This is definitely an issue and I'll look into what we can do. Stay tuned. > > Best, > Ben > > > > -----Original Message----- > > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Lee > > Roberts > > Sent: samedi 6 avril 2019 06:22 > > To: vpp-dev@lists.fd.io > > Subject: [vpp-dev] runtime undefined symbol ibv_cmd_query_port when > > using VPP external rdma-core > > > > I am encountering a runtime error when starting VPP 19.04-rc0 (top-of- > > tree) > > > > after updating to the latest version this morning. > > > > > > > > Apr 5 17:17:38 cl41 /usr/bin/vpp[13913]: dpdk_config:1338: socket-mem > > argument is deprecated > > > > Apr 5 17:17:38 cl41 vpp: /usr/bin/vpp[13913]: dpdk_config:1338: > > socket- mem argument is deprecated > > > > Apr 5 17:17:38 cl41 vpp: /usr/bin/vpp[13913]: dpdk: EAL init args: -c > > ccce -n 8 --in-memory --vdev crypto_null,socket_id=0,max_nb_queue_ > > > > pairs=8 --file-prefix vpp -w 0000:41:00.0 -w 0000:41:00.1 > > --master-lcore 1 > > > > Apr 5 17:17:38 cl41 /usr/bin/vpp[13913]: dpdk: EAL init args: -c ccce > > -n > > 8 --in-memory --vdev crypto_null,socket_id=0,max_nb_queue_pairs > > > > =8 --file-prefix vpp -w 0000:41:00.0 -w 0000:41:00.1 --master-lcore 1 > > > > Apr 5 17:17:39 cl41 vpp: /usr/bin/vpp: symbol lookup error: > > /usr/lib/vpp_plugins/dpdk_plugin.so: undefined symbol: > > ibv_cmd_query_port > > > > Apr 5 17:17:39 cl41 systemd: vpp.service: main process exited, > > code=exited, status=127/n/a > > > > Apr 5 17:17:39 cl41 systemd: Unit vpp.service entered failed state. > > > > Apr 5 17:17:39 cl41 systemd: vpp.service failed. > > > > Apr 5 17:17:44 cl41 systemd: vpp.service holdoff time over, > > scheduling restart. > > > > > > > > I am running on RHEL 7.6 with its version of rdma-core: > > > > > > > > # uname -a > > > > Linux cl41.dpdk.nfv 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 > > UTC > > 2018 x86_64 x86_64 x86_64 GNU/Linux > > > > # rpm -qa | grep rdma-core > > > > rdma-core-17.2-3.el7.x86_64 > > > > rdma-core-devel-17.2-3.el7.x86_64 > > > > # > > > > > > > > I am using a ConnectX-5 adapter, so the MLX5 PMD is being included: > > > > > > > > # make install-ext-deps DPDK_MLX5_PMD=y DPDK_CCP_PMD=y > > > > # make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes > > > > > > > > I have found that if I remove the VPP external "rdma-core" build, then > > VPP will start successfully. > > > > The following changes are used to avoid the issue: > > > > > > > > diff --git a/build/external/Makefile b/build/external/Makefile > > > > index c56e16d..89fe0b9 100644 > > > > --- a/build/external/Makefile > > > > +++ b/build/external/Makefile > > > > @@ -38,17 +38,16 @@ include packages/nasm.mk > > > > include packages/ipsec-mb.mk > > > > include packages/quicly.mk > > > > include packages/dpdk.mk > > > > -include packages/rdma-core.mk > > > > .PHONY: clean > > > > clean: > > > > @rm -rf $(B) $(I) > > > > .PHONY: install > > > > -install: dpdk-install rdma-core-install quicly-install > > > > +install: dpdk-install quicly-install > > > > .PHONY: config > > > > -config: dpdk-config rdma-core-config > > > > +config: dpdk-config > > > > > > ###################################################################### > > #### > > #### > > > > # .deb packaging > > > > > > > > During the VPP build with the above changes, the Red Hat version of > > the ibverbs library > > > > is used: > > > > > > > > @@@@ Configuring vpp in /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > > root/build-vpp-native/vpp @@@@ > > > > -- The C compiler identification is GNU 7.3.1 > > > > -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/cc > > > > -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/cc -- > > works > > > > -- Detecting C compiler ABI info > > > > -- Detecting C compiler ABI info - done > > > > -- Detecting C compile features > > > > -- Detecting C compile features - done > > > > -- Performing Test compiler_flag_march_core_avx2 > > > > -- Performing Test compiler_flag_march_core_avx2 - Success > > > > -- Performing Test compiler_flag_march_skylake_avx512 > > > > -- Performing Test compiler_flag_march_skylake_avx512 - Success > > > > -- Looking for ccache > > > > -- Looking for ccache - found > > > > -- Performing Test compiler_flag_no_address_of_packed_member > > > > -- Performing Test compiler_flag_no_address_of_packed_member - Success > > > > -- Looking for pthread.h > > > > -- Looking for pthread.h - found > > > > -- Looking for pthread_create > > > > -- Looking for pthread_create - not found > > > > -- Check if compiler accepts -pthread > > > > -- Check if compiler accepts -pthread - yes > > > > -- Found Threads: TRUE > > > > -- Performing Test HAVE_MEMFD_CREATE > > > > -- Performing Test HAVE_MEMFD_CREATE - Failed > > > > -- Performing Test HAVE_GETCPU > > > > -- Performing Test HAVE_GETCPU - Failed > > > > -- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.0.2k") > > > > -- The ASM compiler identification is GNU > > > > -- Found assembler: /opt/rh/devtoolset-7/root/bin/cc > > > > -- Looking for libuuid > > > > -- Found uuid in /usr/include > > > > -- dpdk plugin needs libdpdk.a library - found at > > /opt/vpp/external/x86_64/lib/libdpdk.a > > > > -- Found DPDK 19.2.0 in /opt/vpp/external/x86_64/include/dpdk > > > > -- dpdk plugin needs numa library - found at /usr/lib64/libnuma.so > > > > -- dpdk plugin needs libIPSec_MB.a library - found at > > /opt/vpp/external/x86_64/lib/libIPSec_MB.a > > > > -- dpdk plugin needs mnl library - found at /usr/lib64/libmnl.so > > > > -- dpdk plugin needs ibverbs library - found at > > /usr/lib64/libibverbs.so > > > > -- dpdk plugin needs mlx5 library - found at /usr/lib64/libmlx5.so > > > > -- DPDK depends on libmnl, libibverbs, libmlx5 (Mellanox PMD > > requirement) > > > > -- Looking for quicly > > > > -- Found quicly in /opt/vpp/external/x86_64/include > > > > -- rdma plugin needs libibverbs.a library - found at > > /usr/lib64/libibverbs.so > > > > -- rdma plugin needs libmlx5.a library - found at > > /usr/lib64/libmlx5.so > > > > -- Performing Test IBVERBS_COMPILES_CHECK > > > > -- Performing Test IBVERBS_COMPILES_CHECK - Success > > > > -- tlsmbedtls plugin needs mbedtls library - found at > > /usr/lib64/libmbedtls.so > > > > -- tlsmbedtls plugin needs mbedx509 library - found at > > /usr/lib64/libmbedx509.so > > > > -- tlsmbedtls plugin needs mbedcrypto library - found at > > /usr/lib64/libmbedcrypto.so > > > > -- Looking for SSL_set_async_callback > > > > -- Looking for SSL_set_async_callback - not found > > > > -- Found PythonInterp: /usr/bin/python2.7 (found suitable version > > "2.7.5", minimum required is "2.7") > > > > -- Configuration: > > > > VPP version : 19.04-rc0~642-g35c5208 > > > > VPP library version : 19.04 > > > > GIT toplevel dir : /opt/vpp > > > > Build type : > > > > C flags : -Wno-address-of-packed-member -march=corei7 - > > mtune=corei7-avx -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC > > -Werror > > > > Linker flags : -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC > - > > Werror -pie > > > > Host processor : x86_64 > > > > Target processor : x86_64 > > > > Prefix path : /opt/vpp/external/x86_64;/opt/vpp/build- > > root/rpmbuild/vpp-19.04/build-root/install-vpp-native/external > > > > Install prefix : /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > > root/install-vpp-native/vpp > > > > -- Configuring done > > > > -- Generating done > > > > -- Build files have been written to: /opt/vpp/build-root/rpmbuild/vpp- > > 19.04/build-root/build-vpp-native/vpp > > > > @@@@ Building vpp in /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > > root/build-vpp-native/vpp @@@@ > > > > > > > > Without the VPP external "rdma-core", VPP 19.04 will start successfully. > > > > > > > > - Lee Roberts
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12725): https://lists.fd.io/g/vpp-dev/message/12725 Mute This Topic: https://lists.fd.io/mt/30931630/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
