Dave, this is probably an issue we want to fix for VPP 19.04. Right, there is a conflict between internal VPP rdma-core and external (typically distribution package) rdma-core when enabling MLX DPDK support. We compile a very small subset of RDMA core statically to enable VPP RDMA driver. This subset does not include all the symbols needed by MLX DPDK driver, hence the issue you ran into. This is definitely an issue and I'll look into what we can do. Stay tuned.
Best, Ben > -----Original Message----- > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Lee Roberts > Sent: samedi 6 avril 2019 06:22 > To: vpp-dev@lists.fd.io > Subject: [vpp-dev] runtime undefined symbol ibv_cmd_query_port when using > VPP external rdma-core > > I am encountering a runtime error when starting VPP 19.04-rc0 (top-of- > tree) > > after updating to the latest version this morning. > > > > Apr 5 17:17:38 cl41 /usr/bin/vpp[13913]: dpdk_config:1338: socket-mem > argument is deprecated > > Apr 5 17:17:38 cl41 vpp: /usr/bin/vpp[13913]: dpdk_config:1338: socket- > mem argument is deprecated > > Apr 5 17:17:38 cl41 vpp: /usr/bin/vpp[13913]: dpdk: EAL init args: -c > ccce -n 8 --in-memory --vdev crypto_null,socket_id=0,max_nb_queue_ > > pairs=8 --file-prefix vpp -w 0000:41:00.0 -w 0000:41:00.1 --master-lcore 1 > > Apr 5 17:17:38 cl41 /usr/bin/vpp[13913]: dpdk: EAL init args: -c ccce -n > 8 --in-memory --vdev crypto_null,socket_id=0,max_nb_queue_pairs > > =8 --file-prefix vpp -w 0000:41:00.0 -w 0000:41:00.1 --master-lcore 1 > > Apr 5 17:17:39 cl41 vpp: /usr/bin/vpp: symbol lookup error: > /usr/lib/vpp_plugins/dpdk_plugin.so: undefined symbol: ibv_cmd_query_port > > Apr 5 17:17:39 cl41 systemd: vpp.service: main process exited, > code=exited, status=127/n/a > > Apr 5 17:17:39 cl41 systemd: Unit vpp.service entered failed state. > > Apr 5 17:17:39 cl41 systemd: vpp.service failed. > > Apr 5 17:17:44 cl41 systemd: vpp.service holdoff time over, scheduling > restart. > > > > I am running on RHEL 7.6 with its version of rdma-core: > > > > # uname -a > > Linux cl41.dpdk.nfv 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC > 2018 x86_64 x86_64 x86_64 GNU/Linux > > # rpm -qa | grep rdma-core > > rdma-core-17.2-3.el7.x86_64 > > rdma-core-devel-17.2-3.el7.x86_64 > > # > > > > I am using a ConnectX-5 adapter, so the MLX5 PMD is being included: > > > > # make install-ext-deps DPDK_MLX5_PMD=y DPDK_CCP_PMD=y > > # make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes > > > > I have found that if I remove the VPP external "rdma-core" build, then VPP > will start successfully. > > The following changes are used to avoid the issue: > > > > diff --git a/build/external/Makefile b/build/external/Makefile > > index c56e16d..89fe0b9 100644 > > --- a/build/external/Makefile > > +++ b/build/external/Makefile > > @@ -38,17 +38,16 @@ include packages/nasm.mk > > include packages/ipsec-mb.mk > > include packages/quicly.mk > > include packages/dpdk.mk > > -include packages/rdma-core.mk > > .PHONY: clean > > clean: > > @rm -rf $(B) $(I) > > .PHONY: install > > -install: dpdk-install rdma-core-install quicly-install > > +install: dpdk-install quicly-install > > .PHONY: config > > -config: dpdk-config rdma-core-config > > +config: dpdk-config > > > ########################################################################## > #### > > # .deb packaging > > > > During the VPP build with the above changes, the Red Hat version of the > ibverbs library > > is used: > > > > @@@@ Configuring vpp in /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > root/build-vpp-native/vpp @@@@ > > -- The C compiler identification is GNU 7.3.1 > > -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/cc > > -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/cc -- works > > -- Detecting C compiler ABI info > > -- Detecting C compiler ABI info - done > > -- Detecting C compile features > > -- Detecting C compile features - done > > -- Performing Test compiler_flag_march_core_avx2 > > -- Performing Test compiler_flag_march_core_avx2 - Success > > -- Performing Test compiler_flag_march_skylake_avx512 > > -- Performing Test compiler_flag_march_skylake_avx512 - Success > > -- Looking for ccache > > -- Looking for ccache - found > > -- Performing Test compiler_flag_no_address_of_packed_member > > -- Performing Test compiler_flag_no_address_of_packed_member - Success > > -- Looking for pthread.h > > -- Looking for pthread.h - found > > -- Looking for pthread_create > > -- Looking for pthread_create - not found > > -- Check if compiler accepts -pthread > > -- Check if compiler accepts -pthread - yes > > -- Found Threads: TRUE > > -- Performing Test HAVE_MEMFD_CREATE > > -- Performing Test HAVE_MEMFD_CREATE - Failed > > -- Performing Test HAVE_GETCPU > > -- Performing Test HAVE_GETCPU - Failed > > -- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.0.2k") > > -- The ASM compiler identification is GNU > > -- Found assembler: /opt/rh/devtoolset-7/root/bin/cc > > -- Looking for libuuid > > -- Found uuid in /usr/include > > -- dpdk plugin needs libdpdk.a library - found at > /opt/vpp/external/x86_64/lib/libdpdk.a > > -- Found DPDK 19.2.0 in /opt/vpp/external/x86_64/include/dpdk > > -- dpdk plugin needs numa library - found at /usr/lib64/libnuma.so > > -- dpdk plugin needs libIPSec_MB.a library - found at > /opt/vpp/external/x86_64/lib/libIPSec_MB.a > > -- dpdk plugin needs mnl library - found at /usr/lib64/libmnl.so > > -- dpdk plugin needs ibverbs library - found at /usr/lib64/libibverbs.so > > -- dpdk plugin needs mlx5 library - found at /usr/lib64/libmlx5.so > > -- DPDK depends on libmnl, libibverbs, libmlx5 (Mellanox PMD requirement) > > -- Looking for quicly > > -- Found quicly in /opt/vpp/external/x86_64/include > > -- rdma plugin needs libibverbs.a library - found at > /usr/lib64/libibverbs.so > > -- rdma plugin needs libmlx5.a library - found at /usr/lib64/libmlx5.so > > -- Performing Test IBVERBS_COMPILES_CHECK > > -- Performing Test IBVERBS_COMPILES_CHECK - Success > > -- tlsmbedtls plugin needs mbedtls library - found at > /usr/lib64/libmbedtls.so > > -- tlsmbedtls plugin needs mbedx509 library - found at > /usr/lib64/libmbedx509.so > > -- tlsmbedtls plugin needs mbedcrypto library - found at > /usr/lib64/libmbedcrypto.so > > -- Looking for SSL_set_async_callback > > -- Looking for SSL_set_async_callback - not found > > -- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.5", > minimum required is "2.7") > > -- Configuration: > > VPP version : 19.04-rc0~642-g35c5208 > > VPP library version : 19.04 > > GIT toplevel dir : /opt/vpp > > Build type : > > C flags : -Wno-address-of-packed-member -march=corei7 - > mtune=corei7-avx -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror > > Linker flags : -g -O2 -DFORTIFY_SOURCE=2 -fstack-protector -fPIC - > Werror -pie > > Host processor : x86_64 > > Target processor : x86_64 > > Prefix path : /opt/vpp/external/x86_64;/opt/vpp/build- > root/rpmbuild/vpp-19.04/build-root/install-vpp-native/external > > Install prefix : /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > root/install-vpp-native/vpp > > -- Configuring done > > -- Generating done > > -- Build files have been written to: /opt/vpp/build-root/rpmbuild/vpp- > 19.04/build-root/build-vpp-native/vpp > > @@@@ Building vpp in /opt/vpp/build-root/rpmbuild/vpp-19.04/build- > root/build-vpp-native/vpp @@@@ > > > > Without the VPP external "rdma-core", VPP 19.04 will start successfully. > > > > - Lee Roberts
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12724): https://lists.fd.io/g/vpp-dev/message/12724 Mute This Topic: https://lists.fd.io/mt/30931630/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
