Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead
of DPDK_MLX_COMMON_PMD=y.
-Matt
On Mon, May 2, 2022 at 11:28 AM Johnny Martinez <john.marti...@gmail.com>
wrote:
> Hello Matt!
>
> *Thank you very much for the reply!*
>
> Our end goal is to be using Alma Linux but we went ahead and tried it in a
> fresh Ubuntu 20.04 installation with the same result.
>
> It looks like it is unable to compile the *mlx5 drivers*.
>
> *I’ve attached the complete build log for your review.*
>
>
>
> *Build was started with:*
>
> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee
> /root/vpp-build.log*
>
>
>
> As an extra precaution, the vpp.mk content is:
>
>
>
> *MACHINE=$(shell uname -m)*
>
> *vpp_arch = native*
>
> *vpp_root_packages = vpp*
>
> *vpp_debug_TAG_BUILD_TYPE = debug*
>
> *vpp_TAG_BUILD_TYPE = release*
>
> *vpp_clang_TAG_BUILD_TYPE = release*
>
> *vpp_gcov_TAG_BUILD_TYPE = gcov*
>
> *vpp_coverity_TAG_BUILD_TYPE = coverity*
>
> *vpp_dpdk_inc_dir = /usr/include/dpdk/*
>
> *vpp_dpdk_lib_dir = /usr/lib64/*
>
> *vpp_uses_dpdk_mlx4_pmd = yes*
>
> *DPDK_MLX4_PMD=y*
>
> *DPDK_MLX5_PMD=y*
>
> *DPDK_MLX_COMMON_PMD=y*
>
>
>
> The build completes, but there’s an issue with the MLX5 driver. As seen in
> the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried searching
> online, but there’s reference to *glue* with regard to the MLX5, but
> nothing actionable I could find:
>
>
>
> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part of
> “make” (means our build options were correctly interperated):*
>
>
>
> *cmake3 --build
> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core
> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a >
> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>
> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs -lrdma_util
> -lccan -lpthread/' -i
> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc
> >>
> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>
>
> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a
> way to install…?) so is disabled. MLX4 however, is built:*
>
>
>
> *Using 'PKG_CONFIG_PATH' from environment with value:
> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>
> *Run-time dependency libmlx4 found: YES 1.0.39.1*
>
> *Using 'PKG_CONFIG_PATH' from environment with value:
> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>
> *Run-time dependency libibverbs found: YES 1.14.39.1*
>
> *Compiler for C supports arguments -std=c11: YES*
>
> *Compiler for C supports arguments -Wno-strict-prototypes
> -Wstrict-prototypes: YES*
>
> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>
> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>
> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>
> *Header <infiniband/mlx4dv.h> has symbol
> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs:
> YES*
>
> *Header <infiniband/mlx4dv.h> has symbol "MLX4DV_QP_MASK_UAR_MMAP_OFFSET"
> with dependencies libmlx4, libibverbs: YES*
>
> *Checking whether type "struct mlx4_wqe_lso_seg" has member "mss_hdr_size"
> with dependencies libmlx4, libibverbs: YES*
>
> *Configuring mlx4_autoconf.h using configuration*
>
> *Compiler for C supports arguments -std=c11: YES*
>
> *Compiler for C supports arguments -Wno-strict-prototypes
> -Wstrict-prototypes: YES*
>
> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>
> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>
> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>
> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
> "common_mlx5"*
>
> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>
> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>
> *Run-time dependency netcope-common found: NO (tried pkgconfig)*
>
>
>
> *Part 3: Shows multiple missing “internal dependency” issues:*
>
>
>
> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency
> "bus_dpaa"*
>
> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency
> "mempool_dpaa"*
>
> *Message: Disabling kni [drivers/net/kni]: missing internal dependency
> "kni"*
>
> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
> "common_mlx5"*
>
> *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal
> dependency "bus_dpaa"*
>
> *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal
> dependency "common_mlx5"*
>
> *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal
> dependency "common_mlx5"*
>
> *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal dependency
> "common_mlx5"*
>
> *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missing internal dependency
> "common_mlx5"*
>
>
> *I am not able to find a way to enable MLX5.*
>
> --
> John M.
>
>
> On Fri, Apr 29, 2022 at 4:31 PM Matthew Smith <mgsm...@netgate.com> wrote:
>
>>
>> Hi John,
>>
>> Are you using a stock build of VPP from packagecloud or did you build
>> your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5)
>> PMDs by default. If you didn't already, you would need to build VPP with
>> the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and
>> DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to
>> y in build/external/packages/dpdk.mk or you could run something like
>> 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That
>> presumes that you're running on ubuntu or some other debian-based distro,
>> if you're running some other OS maybe substituting pkg-snap or pkg-rpm for
>> the pkg-deb in that command would work for you.
>>
>> -Matt
>>
>>
>> On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez <john.marti...@gmail.com>
>> wrote:
>>
>>> Johnny Martinez <john.marti...@gmail.com>
>>> 9:57 AM (5 hours ago)
>>> to Benoit
>>> *@Ben *- thank you *VERY *much for the detailed response!
>>>
>>> I had *partial *success but almost there - now I am seeing some DPDK
>>> errors and other behavior outlined below.
>>>
>>> On my test device I have 3 interfaces:
>>>
>>> - Management interface (kernel interface, not managed by VPP)
>>> - vpp-wan interface
>>> - vpp-lan interface
>>>
>>> I configureD the interfaces with the following:
>>>
>>> set logging class linux-cp rate-limit 1000 level warn syslog-level notice
>>>
>>>
>>> lcp default netns dataplane
>>>
>>> lcp lcp-sync on
>>>
>>> lcp lcp-auto-subint on
>>>
>>>
>>> # vpp-wan
>>>
>>> vpp# lcp create NetVSC1 host-if vpp-wan
>>>
>>> vpp# set int state NetVSC1 up
>>>
>>> vpp# set int ip address NetVSC1 10.0.2.4/24
>>>
>>> vpp# ip route add 0.0.0.0/0 via 10.0.2.1
>>>
>>>
>>> # vpp-lan
>>>
>>> vpp# lcp create NetVSC0 host-if vpp-lan
>>>
>>> vpp# set int state NetVSC0 up
>>>
>>> vpp# set int ip address NetVSC0 10.0.1.4/24
>>>
>>>
>>> I am now able to see the two interfaces in vpp *HOWEVER *I am not able
>>> to set the MTU size on either interface to 1500.
>>>
>>> vpp# set int mtu 1500 NetVSC0
>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>> vpp# set int mtu 1500 NetVSC1
>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>
>>>
>>> I see the following in the log files using '*journalctl | grep dpdk*'
>>> - we get two seperate DPDK errors. *One seems to be interface specific
>>> and the other when we try to set the MTU size*.
>>>
>>> The *orange *error is every second - whereas the MTU error is only when
>>> we try to set the MTU size.
>>>
>>> *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>> RNDIS reports VF but device not found, retrying*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>> Couldn't find port for VF*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>> RNDIS reports VF but device not found, retrying*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>> Couldn't find port for VF*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>> RNDIS reports VF but device not found, retrying*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0]
>>> rte_eth_dev_set_mtu failed (mtu 1496, rv -95)*
>>>
>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu:
>>> Unsupported (dpdk driver doesn't support MTU change*
>>>
>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>> Couldn't find port for VF*
>>>
>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>> RNDIS reports VF but device not found, retrying*
>>>
>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>> Couldn't find port for VF*
>>>
>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>> RNDIS reports VF but device not found, retrying*
>>>
>>>
>>> Anyone have any recommendation on how to resolve this?
>>>
>>> *I have also tried setting dev XXX and vdev commands in the dpdk{}
>>> block, but this causes the virtual interfaces not to be found (as you have
>>> previously mentioned).*
>>>
>>> *Thank you all in advance for your assistance!*
>>>
>>> --
>>> John M
>>>
>>>
>>> On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith <mgsm...@netgate.com>
>>> wrote:
>>>
>>>> Hi Ben,
>>>>
>>>> Sure, I'll work on an update to the doc.
>>>>
>>>> -Matt
>>>>
>>>>
>>>> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) <bga...@cisco.com>
>>>> wrote:
>>>>
>>>>> Hi Matt, John,
>>>>>
>>>>> This is a great summary!
>>>>>
>>>>> >> The only reference I could find for VPP in Azure was in this link
>>>>> >> but it is very outdated:
>>>>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html
>>>>>
>>>>> > Yes, that doc is way outdated. I advise against following the
>>>>> instructions
>>>>> > in it.
>>>>>
>>>>> Matt or John, would you mind proposing a patch for that? Even a quick
>>>>> & dirty "here are the steps you need to follow" would be great.
>>>>>
>>>>> Best
>>>>> ben
>>>>>
>>>>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21326): https://lists.fd.io/g/vpp-dev/message/21326
Mute This Topic: https://lists.fd.io/mt/90759128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
