Hi John,
I have had some problems on Azure with my build lately similar to the MTU
errors you mentioned. The build that has problems is based on the
commit b0f0f8c8d ("memif: fix memif_process_desc indexing") from the
upstream master branch (so..master branch from 1 or 2 months ago using DPDK
21.11, not a stable release version of VPP). Last week I applied these
recent patches to my build:
739a342b4 dpdk: fix overflow in mtu arithmetic
b6bf5c9c4 dpdk: clear the RTE_MEMPOOL_F_NON_IO
With those patches applied, the problems may have gone away. Netgate's QA
team is still trying to verify if they can reproduce the issue with the
patches applied. It was intermittent to begin with, so its not
straightforward to declare that the issue is resolved.
The build in our previous release, which did not have any issues with
setting MTUs on Azure netvsc interfaces, was based on commit 895def45c
("avf: fix RSS hash key") from the upstream master branch. The DPDK version
used there was 21.08.
-Matt
On Wed, May 18, 2022 at 12:20 PM Johnny Martinez <john.marti...@gmail.com>
wrote:
> Hello Matt,
>
> Can you *please *confirm which DPDK and VPP versions you have that are
> working in Azure?
>
> I am hoping to get a base build with no errors and the ability to set MTU
> size...
>
> *Thank you in advance for your help!!!*
>
> --
> John M.
>
>
> On Mon, May 2, 2022 at 1:42 PM Matthew Smith <mgsm...@netgate.com> wrote:
>
>>
>> Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead
>> of DPDK_MLX_COMMON_PMD=y.
>>
>> -Matt
>>
>>
>> On Mon, May 2, 2022 at 11:28 AM Johnny Martinez <john.marti...@gmail.com>
>> wrote:
>>
>>> Hello Matt!
>>>
>>> *Thank you very much for the reply!*
>>>
>>> Our end goal is to be using Alma Linux but we went ahead and tried it in
>>> a fresh Ubuntu 20.04 installation with the same result.
>>>
>>> It looks like it is unable to compile the *mlx5 drivers*.
>>>
>>> *I’ve attached the complete build log for your review.*
>>>
>>>
>>>
>>> *Build was started with:*
>>>
>>> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb |
>>> tee /root/vpp-build.log*
>>>
>>>
>>>
>>> As an extra precaution, the vpp.mk content is:
>>>
>>>
>>>
>>> *MACHINE=$(shell uname -m)*
>>>
>>> *vpp_arch = native*
>>>
>>> *vpp_root_packages = vpp*
>>>
>>> *vpp_debug_TAG_BUILD_TYPE = debug*
>>>
>>> *vpp_TAG_BUILD_TYPE = release*
>>>
>>> *vpp_clang_TAG_BUILD_TYPE = release*
>>>
>>> *vpp_gcov_TAG_BUILD_TYPE = gcov*
>>>
>>> *vpp_coverity_TAG_BUILD_TYPE = coverity*
>>>
>>> *vpp_dpdk_inc_dir = /usr/include/dpdk/*
>>>
>>> *vpp_dpdk_lib_dir = /usr/lib64/*
>>>
>>> *vpp_uses_dpdk_mlx4_pmd = yes*
>>>
>>> *DPDK_MLX4_PMD=y*
>>>
>>> *DPDK_MLX5_PMD=y*
>>>
>>> *DPDK_MLX_COMMON_PMD=y*
>>>
>>>
>>>
>>> The build completes, but there’s an issue with the MLX5 driver. As seen
>>> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried
>>> searching online, but there’s reference to *glue* with regard to the
>>> MLX5, but nothing actionable I could find:
>>>
>>>
>>>
>>> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part
>>> of “make” (means our build options were correctly interperated):*
>>>
>>>
>>>
>>> *cmake3 --build
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core
>>> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a >
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>>
>>> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs
>>> -lrdma_util -lccan -lpthread/' -i
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc
>>> >>
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>>
>>>
>>> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a
>>> way to install…?) so is disabled. MLX4 however, is built:*
>>>
>>>
>>>
>>> *Using 'PKG_CONFIG_PATH' from environment with value:
>>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>>
>>> *Run-time dependency libmlx4 found: YES 1.0.39.1*
>>>
>>> *Using 'PKG_CONFIG_PATH' from environment with value:
>>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>>
>>> *Run-time dependency libibverbs found: YES 1.14.39.1*
>>>
>>> *Compiler for C supports arguments -std=c11: YES*
>>>
>>> *Compiler for C supports arguments -Wno-strict-prototypes
>>> -Wstrict-prototypes: YES*
>>>
>>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>>
>>> *Header <infiniband/mlx4dv.h> has symbol
>>> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs:
>>> YES*
>>>
>>> *Header <infiniband/mlx4dv.h> has symbol
>>> "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" with dependencies libmlx4, libibverbs: YES*
>>>
>>> *Checking whether type "struct mlx4_wqe_lso_seg" has member
>>> "mss_hdr_size" with dependencies libmlx4, libibverbs: YES*
>>>
>>> *Configuring mlx4_autoconf.h using configuration*
>>>
>>> *Compiler for C supports arguments -std=c11: YES*
>>>
>>> *Compiler for C supports arguments -Wno-strict-prototypes
>>> -Wstrict-prototypes: YES*
>>>
>>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>>
>>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>>> "common_mlx5"*
>>>
>>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>>
>>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>>
>>> *Run-time dependency netcope-common found: NO (tried pkgconfig)*
>>>
>>>
>>>
>>> *Part 3: Shows multiple missing “internal dependency” issues:*
>>>
>>>
>>>
>>> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency
>>> "bus_dpaa"*
>>>
>>> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency
>>> "mempool_dpaa"*
>>>
>>> *Message: Disabling kni [drivers/net/kni]: missing internal dependency
>>> "kni"*
>>>
>>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>>> "common_mlx5"*
>>>
>>> *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal
>>> dependency "bus_dpaa"*
>>>
>>> *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>>
>>> *I am not able to find a way to enable MLX5.*
>>>
>>> --
>>> John M.
>>>
>>>
>>> On Fri, Apr 29, 2022 at 4:31 PM Matthew Smith <mgsm...@netgate.com>
>>> wrote:
>>>
>>>>
>>>> Hi John,
>>>>
>>>> Are you using a stock build of VPP from packagecloud or did you build
>>>> your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5)
>>>> PMDs by default. If you didn't already, you would need to build VPP with
>>>> the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and
>>>> DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to
>>>> y in build/external/packages/dpdk.mk or you could run something like
>>>> 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That
>>>> presumes that you're running on ubuntu or some other debian-based distro,
>>>> if you're running some other OS maybe substituting pkg-snap or pkg-rpm for
>>>> the pkg-deb in that command would work for you.
>>>>
>>>> -Matt
>>>>
>>>>
>>>> On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez <
>>>> john.marti...@gmail.com> wrote:
>>>>
>>>>> Johnny Martinez <john.marti...@gmail.com>
>>>>> 9:57 AM (5 hours ago)
>>>>> to Benoit
>>>>> *@Ben *- thank you *VERY *much for the detailed response!
>>>>>
>>>>> I had *partial *success but almost there - now I am seeing some DPDK
>>>>> errors and other behavior outlined below.
>>>>>
>>>>> On my test device I have 3 interfaces:
>>>>>
>>>>> - Management interface (kernel interface, not managed by VPP)
>>>>> - vpp-wan interface
>>>>> - vpp-lan interface
>>>>>
>>>>> I configureD the interfaces with the following:
>>>>>
>>>>> set logging class linux-cp rate-limit 1000 level warn syslog-level
>>>>> notice
>>>>>
>>>>>
>>>>> lcp default netns dataplane
>>>>>
>>>>> lcp lcp-sync on
>>>>>
>>>>> lcp lcp-auto-subint on
>>>>>
>>>>>
>>>>> # vpp-wan
>>>>>
>>>>> vpp# lcp create NetVSC1 host-if vpp-wan
>>>>>
>>>>> vpp# set int state NetVSC1 up
>>>>>
>>>>> vpp# set int ip address NetVSC1 10.0.2.4/24
>>>>>
>>>>> vpp# ip route add 0.0.0.0/0 via 10.0.2.1
>>>>>
>>>>>
>>>>> # vpp-lan
>>>>>
>>>>> vpp# lcp create NetVSC0 host-if vpp-lan
>>>>>
>>>>> vpp# set int state NetVSC0 up
>>>>>
>>>>> vpp# set int ip address NetVSC0 10.0.1.4/24
>>>>>
>>>>>
>>>>> I am now able to see the two interfaces in vpp *HOWEVER *I am not
>>>>> able to set the MTU size on either interface to 1500.
>>>>>
>>>>> vpp# set int mtu 1500 NetVSC0
>>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>>> vpp# set int mtu 1500 NetVSC1
>>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>>>
>>>>>
>>>>> I see the following in the log files using '*journalctl | grep dpdk*'
>>>>> - we get two seperate DPDK errors. *One seems to be interface
>>>>> specific and the other when we try to set the MTU size*.
>>>>>
>>>>> The *orange *error is every second - whereas the MTU error is only
>>>>> when we try to set the MTU size.
>>>>>
>>>>> *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0]
>>>>> rte_eth_dev_set_mtu failed (mtu 1496, rv -95)*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu:
>>>>> Unsupported (dpdk driver doesn't support MTU change*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>>
>>>>> Anyone have any recommendation on how to resolve this?
>>>>>
>>>>> *I have also tried setting dev XXX and vdev commands in the dpdk{}
>>>>> block, but this causes the virtual interfaces not to be found (as you have
>>>>> previously mentioned).*
>>>>>
>>>>> *Thank you all in advance for your assistance!*
>>>>>
>>>>> --
>>>>> John M
>>>>>
>>>>>
>>>>> On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith <mgsm...@netgate.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ben,
>>>>>>
>>>>>> Sure, I'll work on an update to the doc.
>>>>>>
>>>>>> -Matt
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) <
>>>>>> bga...@cisco.com> wrote:
>>>>>>
>>>>>>> Hi Matt, John,
>>>>>>>
>>>>>>> This is a great summary!
>>>>>>>
>>>>>>> >> The only reference I could find for VPP in Azure was in this link
>>>>>>> >> but it is very outdated:
>>>>>>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html
>>>>>>>
>>>>>>> > Yes, that doc is way outdated. I advise against following the
>>>>>>> instructions
>>>>>>> > in it.
>>>>>>>
>>>>>>> Matt or John, would you mind proposing a patch for that? Even a
>>>>>>> quick & dirty "here are the steps you need to follow" would be great.
>>>>>>>
>>>>>>> Best
>>>>>>> ben
>>>>>>>
>>>>>>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21447): https://lists.fd.io/g/vpp-dev/message/21447
Mute This Topic: https://lists.fd.io/mt/90759128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
