Hello Matt,
*Here is some more detail that may be useful:*
*Loaded modules (loaded by doing `* *modprobe uio_hv_generic`):*
*[root@demo001 vpp]# lsmod | grep uio*
*uio_pci_generic 16384 0*
*uio_hv_generic 16384 12*
*hv_vmbus 143360 9
hv_balloon,hv_utils,uio_hv_generic,hv_netvsc,hid_hyperv,hv_storvsc,hyperv_keyboard,pci_hyperv,hyperv_fb*
*uio 20480 27 uio_hv_generic,cnic,uio_pci_generic*
*PCI Devices:*
*[root@demo001 vpp]# lspci | grep Ethernet*
*2ab4:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function] (rev 80)*
*8213:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function] (rev 80)*
*e2ca:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function] (rev 80)*
*Hardware:*
*[root@demo001 vpp]# lshw -class network -businfo*
*Bus info Device Class Description*
*========================================================*
*pci@2ab4:00:02.0 eth5 network MT27710 Family [ConnectX-4 Lx
Virtual Function]*
*pci@8213:00:02.0 eth3 network MT27710 Family [ConnectX-4 Lx
Virtual Function]*
*pci@e2ca:00:02.0 eth4 network MT27710 Family [ConnectX-4 Lx
Virtual Function]*
* eth0 network Ethernet interface*
*Kernel: (We’ve tried with 4.18, 4.20, 5.11 and 5.16)*
*NOTE: kernel 4.20 is end of life*
*[root@demo001 vpp]# uname -a*
*Linux demo001 5.16.13-1.el8.elrepo.x86_64 #1 SMP PREEMPT Tue Mar 8
08:33:30 EST 2022 x86_64 x86_64 x86_64 GNU/Linux*
*Interface states:*
*[root@demo001 ~]# ip addr | grep "state"*
*1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000*
*2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
group default qlen 1000*
*5: eth3: <BROADCAST,MULTICAST,SLAVE> mtu 1500 qdisc mq state DOWN group
default qlen 1000*
*6: eth4: <BROADCAST,MULTICAST,SLAVE> mtu 1500 qdisc mq master eth0
state DOWN group default qlen 1000*
*7: eth5: <BROADCAST,MULTICAST,SLAVE> mtu 1500 qdisc mq state DOWN group
default qlen 1000*
*WITHOUT `dev` config options in dpdk{} section of startup:*
*May 03 11:19:54 demo001 vpp[2154]: vpp[2154]: vlib_vmbus_bind_to_uio:
Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host
interface eth0 is up*
*May 03 11:19:54 demo001 vpp[2154]: vlib_vmbus_bind_to_uio: Skipping VMBUS
device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up*
*May 03 11:19:54 demo001 vpp[2154]: vpp[2154]: dpdk: EAL init args:
--in-memory --no-telemetry --file-prefix vpp -b
000d3a8a-a89d-000d-3a8a-a89d000d3a8a*
*May 03 11:19:54 demo001 vpp[2154]: dpdk: EAL init args: --in-memory
--no-telemetry --file-prefix vpp -b 000d3a8a-a89d-000d-3a8a-a89d000d3a8a*
*May 03 11:19:55 demo001 kernel: mlx5_core 2ab4:00:02.0 eth5: Link up*
*May 03 11:19:55 demo001 NetworkManager[1047]: <info> [1651576795.1466]
device (eth5): carrier: link connected*
*May 03 11:19:55 demo001 kernel: mlx5_core 8213:00:02.0 eth3: Link up*
*May 03 11:19:55 demo001 NetworkManager[1047]: <info> [1651576795.3444]
device (eth3): carrier: link connected*
*May 03 11:19:55 demo001 kernel: mlx5_core e2ca:00:02.0 eth4: Link up*
*May 03 11:19:55 demo001 kernel: hv_netvsc
000d3a8a-a89d-000d-3a8a-a89d000d3a8a eth0: Data path switched to VF: eth4*
*May 03 11:19:55 demo001 NetworkManager[1047]: <info> [1651576795.5913]
device (eth4): carrier: link connected*
*May 03 11:19:55 demo001 vnet[2154]: dpdk: Interface error -22: Unknown
error -22*
*May 03 11:19:55 demo001 vnet[2154]: dpdk: Interface error -22: Unknown
error -22*
*May 03 11:19:55 demo001 vnet[2154]: dpdk: [2] setup failed Errors:*
* rte_eth_tx_queue_setup[port:2,
errno:-22]: Unknown error -22*
* rte_eth_rx_queue_setup[port:2,
errno:-22]: Unknown error -22*
*May 03 11:19:55 demo001 vnet[2154]: received signal SIGSEGV, PC
0x7fcdaab1512c, faulting address 0x0*
*May 03 11:19:55 demo001 vnet[2154]: #0 0x00007fcdab61f9d7 0x7fcdab61f9d7*
*May 03 11:19:55 demo001 vnet[2154]: #1 0x00007fcdaaf59c20 0x7fcdaaf59c20*
*May 03 11:19:55 demo001 vnet[2154]: #2 0x00007fcdaab1512c 0x7fcdaab1512c*
*May 03 11:19:55 demo001 vnet[2154]: #3 0x00007fcdaab16bfe va_format +
0xee*
*May 03 11:19:55 demo001 vnet[2154]: #4 0x00007fcdaab18b04 format + 0x84*
*May 03 11:19:55 demo001 vnet[2154]: #5 0x00007fcdaab1488c 0x7fcdaab1488c*
*May 03 11:19:55 demo001 vnet[2154]: #6 0x00007fcdaab16bfe va_format +
0xee*
*May 03 11:19:55 demo001 vnet[2154]: #7 0x00007fcdab5c39f8 vlib_log +
0x2a8*
*May 03 11:19:55 demo001 vnet[2154]: #8 0x00007fcd45389305 0x7fcd45389305*
*May 03 11:19:55 demo001 vnet[2154]: #9 0x00007fcd45399382 0x7fcd45399382*
*May 03 11:19:55 demo001 vnet[2154]: #10 0x00007fcd4539e0a6 0x7fcd4539e0a6*
*May 03 11:19:55 demo001 vnet[2154]: #11 0x00007fcdab5c4ac6 0x7fcdab5c4ac6*
*May 03 11:19:55 demo001 vnet[2154]: #12 0x00007fcdaab2ce10 0x7fcdaab2ce10*
*May 03 11:19:55 demo001 vpp[2154]: Failed to save post-mortem API trace to
/tmp/api_post_mortem.2154*
*May 03 11:19:55 demo001 systemd[1]: vpp.service: Main process exited,
code=killed, status=6/ABRT*
*May 03 11:19:55 demo001 systemd[1]: vpp.service: Failed with result
'signal'.*
*-- Subject: Unit failed*
*WITH `dev` config options in dpdk{} section of startup:*
*May 03 11:21:15 demo001 vpp[2305]: vpp[2305]: dpdk: EAL init args:
--in-memory --no-telemetry --file-prefix vpp -a 2ab4:00:02.0 -a
8213:00:02.0 -a e2ca:00:02.0*
*May 03 11:21:15 demo001 vpp[2305]: dpdk: EAL init args: --in-memory
--no-telemetry --file-prefix vpp -a 2ab4:00:02.0 -a 8213:00:02.0 -a
e2ca:00:02.0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Interface error -22: Unknown
error -22*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Interface error -22: Unknown
error -22*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: [2] setup failed Errors:*
* rte_eth_tx_queue_setup[port:2,
errno:-22]: Unknown error -22*
* rte_eth_rx_queue_setup[port:2,
errno:-22]: Unknown error -22*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: [3] Device net_netvsc
blacklisted. Skipping...*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: [4] Device net_netvsc
blacklisted. Skipping...*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Detected CPU lcores: 4*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Detected NUMA nodes: 1*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Detected static linkage of
DPDK*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Selected IOVA mode 'PA'*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: No free 1048576 kB
hugepages reported on node 0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Probe PCI driver: mlx5_pci
(15b3:1016) device: 2ab4:00:02.0 (socket 0)*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: mlx5_net: No available register
for sampler.*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Probe PCI driver: mlx5_pci
(15b3:1016) device: 8213:00:02.0 (socket 0)*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: mlx5_net: No available register
for sampler.*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Probe PCI driver: mlx5_pci
(15b3:1016) device: e2ca:00:02.0 (socket 0)*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: mlx5_net: No available register
for sampler.*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: hn_vf_attach(): found matching
VF port 0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: hn_vf_attach(): found matching
VF port 1*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: VFIO support not
initialized*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: EAL: Couldn't map new region for
DMA*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Frame size (65553) > device max
frame size (65536) for port_id 2*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Invalid Tx queue_id=0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Invalid Tx queue_id=0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Invalid Rx queue_id=0*
*May 03 11:21:16 demo001 vnet[2305]: dpdk: Invalid Rx queue_id=0*
*WITH startup.conf dev section:*
*[root@demo001 ~]# vppctl show int*
* Name Idx State MTU (L3/IP4/IP6/MPLS)
Counter Count*
*Ethernet0/2/0 1 error 9000/0/0/0*
*local0 0 down 0/0/0/0*
*WITHOUT startup.conf dev section (won’t work, because vpp daemon SIGABRT):*
*connect: Connection refused*
*Sections played with in startup.conf (hashed vs. unhashed as shown
above):**#dpdk
{*
*# dev 2ab4:00:02.0*
*# dev 8213:00:02.0*
*# dev e2ca:00:02.0*
*#}*
*Trying with interfaces up (not expecting it to work):**May 03 11:26:29
demo001 vpp[1488]: vpp[1488]: dpdk: EAL init args: --in-memory
--no-telemetry --file-prefix vpp*
*May 03 11:26:29 demo001 vpp[1488]: dpdk: EAL init args: --in-memory
--no-telemetry --file-prefix vpp*
*May 03 11:26:30 demo001 vnet[1488]: dpdk: Interface Ethernet0/2/0 error
-22: Unknown error -22*
*May 03 11:26:30 demo001 vnet[1488]: dpdk: Interface Ethernet0/2/0 error
-22: Unknown error -22*
*May 03 11:26:30 demo001 vnet[1488]: dpdk: [0] setup failed Errors:*
* rte_eth_tx_queue_setup[port:0,
errno:-22]: Unknown error -22*
* rte_eth_rx_queue_setup[port:0,
errno:-22]: Unknown error -22*
*May 03 11:26:30 demo001 vnet[1488]: vlib_register_node:383: more than one
node named `Ethernet0/2/0-tx'*
*May 03 11:26:30 demo001 systemd[1]: vpp.service: Main process exited,
code=exited, status=1/FAILURE*
*May 03 11:26:30 demo001 systemd[1]: vpp.service: Failed with result
'exit-code'.*
*-- Subject: Unit failed*
*NOTE:* Was not expecting *vlib_register_node:383: more than one node named
`Ethernet0/2/0-tx'. This is true, when interfaces are UP regardless of UIO
module loaded.*
*Below is the output of the mlx5 driver when it compiled - several items
returned "NO".*
Run-time dependency libmlx5 found: YES 1.22.39.1
Header <infiniband/verbs.h> has symbol "IBV_ACCESS_RELAXED_ORDERING " with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP" with dependencies libmlx5, libibverbs:
YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_CQ_INIT_ATTR_FLAGS_CQE_PAD"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_create_flow_action_packet_reformat" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/verbs.h> has symbol "IBV_FLOW_SPEC_MPLS" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/verbs.h> has symbol "IBV_WQ_FLAGS_PCI_WRITE_END_PADDING"
with dependencies libmlx5, libibverbs: YES
*Header <infiniband/verbs.h> has symbol "IBV_WQ_FLAG_RX_END_PADDING" with
dependencies libmlx5, libibverbs: NOHeader <infiniband/mlx5dv.h> has symbol
"mlx5dv_query_devx_port" with dependencies libmlx5, libibverbs: NO*
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_query_port" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_action_create_dest_ib_port" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_devx_obj_create" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_FLOW_ACTION_COUNTERS_DEVX"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_FLOW_ACTION_DEFAULT_MISS"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_devx_obj_query_async" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_devx_qp_query" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_pp_alloc" with dependencies
libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_action_create_dest_devx_tir" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_devx_get_event" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_action_create_flow_meter" with dependencies libmlx5, libibverbs:
YES
Header <infiniband/mlx5dv.h> has symbol "MLX5_MMAP_GET_NC_PAGES_CMD" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_DR_DOMAIN_TYPE_NIC_RX" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5DV_DR_DOMAIN_TYPE_FDB" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_dr_action_create_push_vlan"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_alloc_var" with
dependencies libmlx5, libibverbs: YES
*Header <infiniband/mlx5dv.h> has symbol "MLX5_OPCODE_ENHANCED_MPSW" with
dependencies libmlx5, libibverbs: NOHeader <infiniband/mlx5dv.h> has symbol
"MLX5_OPCODE_SEND_EN" with dependencies libmlx5, libibverbs: NOHeader
<infiniband/mlx5dv.h> has symbol "MLX5_OPCODE_WAIT" with dependencies
libmlx5, libibverbs: NOHeader <infiniband/mlx5dv.h> has symbol
"MLX5_OPCODE_ACCESS_ASO" with dependencies libmlx5, libibverbs: NO*
Header <linux/ethtool.h> has symbol "SUPPORTED_40000baseKR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_40000baseCR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_40000baseSR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_40000baseLR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_56000baseKR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_56000baseCR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_56000baseSR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol "SUPPORTED_56000baseLR4_Full" with
dependencies libmlx5, libibverbs: YES
Header <linux/ethtool.h> has symbol
"ETHTOOL_LINK_MODE_25000baseCR_Full_BIT" with dependencies libmlx5,
libibverbs: YES
Header <linux/ethtool.h> has symbol
"ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT" with dependencies libmlx5,
libibverbs: YES
Header <linux/ethtool.h> has symbol
"ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT" with dependencies libmlx5,
libibverbs: YES
Header <linux/if_link.h> has symbol "IFLA_NUM_VF" with dependencies
libmlx5, libibverbs: YES
Header <linux/if_link.h> has symbol "IFLA_EXT_MASK" with dependencies
libmlx5, libibverbs: YES
Header <linux/if_link.h> has symbol "IFLA_PHYS_SWITCH_ID" with dependencies
libmlx5, libibverbs: YES
Header <linux/if_link.h> has symbol "IFLA_PHYS_PORT_NAME" with dependencies
libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NL_NLDEV" with dependencies
libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_CMD_GET" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_CMD_PORT_GET" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_ATTR_DEV_INDEX" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_ATTR_DEV_NAME" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_ATTR_PORT_INDEX" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_ATTR_PORT_STATE" with
dependencies libmlx5, libibverbs: YES
Header <rdma/rdma_netlink.h> has symbol "RDMA_NLDEV_ATTR_NDEV_INDEX" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_dump_dr_domain" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_action_create_flow_sampler" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_domain_set_reclaim_device_memory" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_action_create_dest_array" with dependencies libmlx5, libibverbs:
YES
Header <linux/devlink.h> has symbol "DEVLINK_GENL_NAME" with dependencies
libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_dr_action_create_aso" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/verbs.h> has symbol "INFINIBAND_VERBS_H" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "MLX5_WQE_UMR_CTRL_FLAG_INLINE"
with dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol "mlx5dv_dump_dr_rule" with
dependencies libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"MLX5DV_DR_ACTION_FLAGS_ASO_CT_DIRECTION_INITIATOR" with dependencies
libmlx5, libibverbs: YES
Header <infiniband/mlx5dv.h> has symbol
"mlx5dv_dr_domain_allow_duplicate_rules" with dependencies libmlx5,
libibverbs: YES
Header <infiniband/verbs.h> has symbol "ibv_reg_mr_iova" with dependencies
libmlx5, libibverbs: YES
Header <infiniband/verbs.h> has symbol "ibv_import_device" with
dependencies libmlx5, libibverbs: YES
Checking whether type "struct mlx5dv_sw_parsing_caps" has member
"sw_parsing_offloads" with dependencies libmlx5, libibverbs: YES
*Checking whether type "struct ibv_counter_set_init_attr" has member
"counter_set_id" with dependencies libmlx5, libibverbs: NO*
Checking whether type "struct ibv_counters_init_attr" has member
"comp_mask" with dependencies libmlx5, libibverbs: YES
Checking whether type "struct mlx5dv_devx_uar" has member "mmap_off" with
dependencies libmlx5, libibverbs: YES
--
Johnny
On Mon, May 2, 2022 at 7:25 PM John Martinez via lists.fd.io <john.martinez=
gmail....@lists.fd.io> wrote:
> Hello Matt,
>
> Thank you again for the response.
>
> So we were able to compile but now are unable to start VPP.
>
> We are seeing the following errors:
>
> I rebuilt, and got:
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: perfmon: skipping source
> 'intel-uncore' - intel_uncore_init: no uncore units found
>
> May 02 20:20:24 demo001 vpp[2161]: perfmon: skipping source 'intel-uncore'
> - intel_uncore_init: no uncore units found
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:617: Could
> not initialize TLS CA certificates
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_mbedtls_init:652: failed
> to initialize TLS CA chain
>
> May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:617: Could not
> initialize TLS CA certificates
>
> May 02 20:20:24 demo001 vpp[2161]: tls_mbedtls_init:652: failed to
> initialize TLS CA chain
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:1083:
> Could not initialize TLS CA certificates
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_openssl_init:1206:
> failed to initialize TLS CA chain
>
> May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:1083: Could not
> initialize TLS CA certificates
>
> May 02 20:20:24 demo001 vpp[2161]: tls_openssl_init:1206: failed to
> initialize TLS CA chain
>
> May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: vlib_vmbus_bind_to_uio:
> Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host
> interface eth0 is up
>
> May 02 20:20:24 demo001 vpp[2161]: vlib_vmbus_bind_to_uio: Skipping VMBUS
> device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up
>
> May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error
> -22: Unknown error -22
>
> May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error
> -22: Unknown error -22
>
> May 02 20:20:24 demo001 vnet[2161]: dpdk: [0] setup failed Errors:
>
> rte_eth_tx_queue_setup[port:0,
> errno:-22]: Unknown error -22
>
> rte_eth_rx_queue_setup[port:0,
> errno:-22]: Unknown error -22
>
> May 02 20:20:25 demo001 vnet[2161]: received signal SIGSEGV, PC
> 0x7f4ba079c12c, faulting address 0xdd7
>
> May 02 20:20:25 demo001 vnet[2161]: #0 0x00007f4ba12a69d7 0x7f4ba12a69d7
>
> May 02 20:20:25 demo001 vnet[2161]: #1 0x00007f4ba0be0c20 0x7f4ba0be0c20
>
> May 02 20:20:25 demo001 vnet[2161]: #2 0x00007f4ba079c12c 0x7f4ba079c12c
>
> May 02 20:20:25 demo001 vnet[2161]: #3 0x00007f4ba079dbfe va_format + 0xee
>
> May 02 20:20:25 demo001 vnet[2161]: #4 0x00007f4ba079fb04 format + 0x84
>
> May 02 20:20:25 demo001 vnet[2161]: #5 0x00007f4ba079b88c 0x7f4ba079b88c
>
> May 02 20:20:25 demo001 vnet[2161]: #6 0x00007f4ba079dbfe va_format + 0xee
>
> May 02 20:20:25 demo001 vnet[2161]: #7 0x00007f4ba124a9f8 vlib_log + 0x2a8
>
> May 02 20:20:25 demo001 vnet[2161]: #8 0x00007f4b5b160305 0x7f4b5b160305
>
> May 02 20:20:25 demo001 vnet[2161]: #9 0x00007f4b5b170382 0x7f4b5b170382
>
> May 02 20:20:25 demo001 vnet[2161]: #10 0x00007f4b5b1750a6 0x7f4b5b1750a6
>
> May 02 20:20:25 demo001 vnet[2161]: #11 0x00007f4ba124bac6 0x7f4ba124bac6
>
> May 02 20:20:25 demo001 vnet[2161]: #12 0x00007f4ba07b3e10 0x7f4ba07b3e10
>
> May 02 20:20:25 demo001 vpp[2161]: Failed to save post-mortem API trace to
> /tmp/api_post_mortem.2161
>
> May 02 20:20:25 demo001 systemd[1]: vpp.service: Main process exited,
> code=killed, status=6/ABRT
>
> May 02 20:20:25 demo001 systemd[1]: vpp.service: Failed with result
> 'signal'.
>
> -- Subject: Unit failed
>
>
>
>
>
> This time, vpp doesn’t even start up due to the above errors.
>
>
>
> *Is there a specific kernel version you used?*
>
> --
> John M.
>
>
> On Mon, May 2, 2022 at 1:42 PM Matthew Smith <mgsm...@netgate.com> wrote:
>
>>
>> Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead
>> of DPDK_MLX_COMMON_PMD=y.
>>
>> -Matt
>>
>>
>> On Mon, May 2, 2022 at 11:28 AM Johnny Martinez <john.marti...@gmail.com>
>> wrote:
>>
>>> Hello Matt!
>>>
>>> *Thank you very much for the reply!*
>>>
>>> Our end goal is to be using Alma Linux but we went ahead and tried it in
>>> a fresh Ubuntu 20.04 installation with the same result.
>>>
>>> It looks like it is unable to compile the *mlx5 drivers*.
>>>
>>> *I’ve attached the complete build log for your review.*
>>>
>>>
>>>
>>> *Build was started with:*
>>>
>>> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb |
>>> tee /root/vpp-build.log*
>>>
>>>
>>>
>>> As an extra precaution, the vpp.mk content is:
>>>
>>>
>>>
>>> *MACHINE=$(shell uname -m)*
>>>
>>> *vpp_arch = native*
>>>
>>> *vpp_root_packages = vpp*
>>>
>>> *vpp_debug_TAG_BUILD_TYPE = debug*
>>>
>>> *vpp_TAG_BUILD_TYPE = release*
>>>
>>> *vpp_clang_TAG_BUILD_TYPE = release*
>>>
>>> *vpp_gcov_TAG_BUILD_TYPE = gcov*
>>>
>>> *vpp_coverity_TAG_BUILD_TYPE = coverity*
>>>
>>> *vpp_dpdk_inc_dir = /usr/include/dpdk/*
>>>
>>> *vpp_dpdk_lib_dir = /usr/lib64/*
>>>
>>> *vpp_uses_dpdk_mlx4_pmd = yes*
>>>
>>> *DPDK_MLX4_PMD=y*
>>>
>>> *DPDK_MLX5_PMD=y*
>>>
>>> *DPDK_MLX_COMMON_PMD=y*
>>>
>>>
>>>
>>> The build completes, but there’s an issue with the MLX5 driver. As seen
>>> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried
>>> searching online, but there’s reference to *glue* with regard to the
>>> MLX5, but nothing actionable I could find:
>>>
>>>
>>>
>>> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part
>>> of “make” (means our build options were correctly interperated):*
>>>
>>>
>>>
>>> *cmake3 --build
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core
>>> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a >
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>>
>>> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs
>>> -lrdma_util -lccan -lpthread/' -i
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc
>>> >>
>>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>>
>>>
>>> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a
>>> way to install…?) so is disabled. MLX4 however, is built:*
>>>
>>>
>>>
>>> *Using 'PKG_CONFIG_PATH' from environment with value:
>>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>>
>>> *Run-time dependency libmlx4 found: YES 1.0.39.1*
>>>
>>> *Using 'PKG_CONFIG_PATH' from environment with value:
>>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>>
>>> *Run-time dependency libibverbs found: YES 1.14.39.1*
>>>
>>> *Compiler for C supports arguments -std=c11: YES*
>>>
>>> *Compiler for C supports arguments -Wno-strict-prototypes
>>> -Wstrict-prototypes: YES*
>>>
>>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>>
>>> *Header <infiniband/mlx4dv.h> has symbol
>>> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs:
>>> YES*
>>>
>>> *Header <infiniband/mlx4dv.h> has symbol
>>> "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" with dependencies libmlx4, libibverbs: YES*
>>>
>>> *Checking whether type "struct mlx4_wqe_lso_seg" has member
>>> "mss_hdr_size" with dependencies libmlx4, libibverbs: YES*
>>>
>>> *Configuring mlx4_autoconf.h using configuration*
>>>
>>> *Compiler for C supports arguments -std=c11: YES*
>>>
>>> *Compiler for C supports arguments -Wno-strict-prototypes
>>> -Wstrict-prototypes: YES*
>>>
>>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>>
>>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>>
>>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>>> "common_mlx5"*
>>>
>>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>>
>>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>>
>>> *Run-time dependency netcope-common found: NO (tried pkgconfig)*
>>>
>>>
>>>
>>> *Part 3: Shows multiple missing “internal dependency” issues:*
>>>
>>>
>>>
>>> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency
>>> "bus_dpaa"*
>>>
>>> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency
>>> "mempool_dpaa"*
>>>
>>> *Message: Disabling kni [drivers/net/kni]: missing internal dependency
>>> "kni"*
>>>
>>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>>> "common_mlx5"*
>>>
>>> *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal
>>> dependency "bus_dpaa"*
>>>
>>> *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>> *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missing internal
>>> dependency "common_mlx5"*
>>>
>>>
>>> *I am not able to find a way to enable MLX5.*
>>>
>>> --
>>> John M.
>>>
>>>
>>> On Fri, Apr 29, 2022 at 4:31 PM Matthew Smith <mgsm...@netgate.com>
>>> wrote:
>>>
>>>>
>>>> Hi John,
>>>>
>>>> Are you using a stock build of VPP from packagecloud or did you build
>>>> your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5)
>>>> PMDs by default. If you didn't already, you would need to build VPP with
>>>> the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and
>>>> DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to
>>>> y in build/external/packages/dpdk.mk or you could run something like
>>>> 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That
>>>> presumes that you're running on ubuntu or some other debian-based distro,
>>>> if you're running some other OS maybe substituting pkg-snap or pkg-rpm for
>>>> the pkg-deb in that command would work for you.
>>>>
>>>> -Matt
>>>>
>>>>
>>>> On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez <
>>>> john.marti...@gmail.com> wrote:
>>>>
>>>>> Johnny Martinez <john.marti...@gmail.com>
>>>>> 9:57 AM (5 hours ago)
>>>>> to Benoit
>>>>> *@Ben *- thank you *VERY *much for the detailed response!
>>>>>
>>>>> I had *partial *success but almost there - now I am seeing some DPDK
>>>>> errors and other behavior outlined below.
>>>>>
>>>>> On my test device I have 3 interfaces:
>>>>>
>>>>> - Management interface (kernel interface, not managed by VPP)
>>>>> - vpp-wan interface
>>>>> - vpp-lan interface
>>>>>
>>>>> I configureD the interfaces with the following:
>>>>>
>>>>> set logging class linux-cp rate-limit 1000 level warn syslog-level
>>>>> notice
>>>>>
>>>>>
>>>>> lcp default netns dataplane
>>>>>
>>>>> lcp lcp-sync on
>>>>>
>>>>> lcp lcp-auto-subint on
>>>>>
>>>>>
>>>>> # vpp-wan
>>>>>
>>>>> vpp# lcp create NetVSC1 host-if vpp-wan
>>>>>
>>>>> vpp# set int state NetVSC1 up
>>>>>
>>>>> vpp# set int ip address NetVSC1 10.0.2.4/24
>>>>>
>>>>> vpp# ip route add 0.0.0.0/0 via 10.0.2.1
>>>>>
>>>>>
>>>>> # vpp-lan
>>>>>
>>>>> vpp# lcp create NetVSC0 host-if vpp-lan
>>>>>
>>>>> vpp# set int state NetVSC0 up
>>>>>
>>>>> vpp# set int ip address NetVSC0 10.0.1.4/24
>>>>>
>>>>>
>>>>> I am now able to see the two interfaces in vpp *HOWEVER *I am not
>>>>> able to set the MTU size on either interface to 1500.
>>>>>
>>>>> vpp# set int mtu 1500 NetVSC0
>>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>>> vpp# set int mtu 1500 NetVSC1
>>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>>>
>>>>>
>>>>> I see the following in the log files using '*journalctl | grep dpdk*'
>>>>> - we get two seperate DPDK errors. *One seems to be interface
>>>>> specific and the other when we try to set the MTU size*.
>>>>>
>>>>> The *orange *error is every second - whereas the MTU error is only
>>>>> when we try to set the MTU size.
>>>>>
>>>>> *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0]
>>>>> rte_eth_dev_set_mtu failed (mtu 1496, rv -95)*
>>>>>
>>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu:
>>>>> Unsupported (dpdk driver doesn't support MTU change*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>>> Couldn't find port for VF*
>>>>>
>>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>>> RNDIS reports VF but device not found, retrying*
>>>>>
>>>>>
>>>>> Anyone have any recommendation on how to resolve this?
>>>>>
>>>>> *I have also tried setting dev XXX and vdev commands in the dpdk{}
>>>>> block, but this causes the virtual interfaces not to be found (as you have
>>>>> previously mentioned).*
>>>>>
>>>>> *Thank you all in advance for your assistance!*
>>>>>
>>>>> --
>>>>> John M
>>>>>
>>>>>
>>>>> On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith <mgsm...@netgate.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ben,
>>>>>>
>>>>>> Sure, I'll work on an update to the doc.
>>>>>>
>>>>>> -Matt
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) <
>>>>>> bga...@cisco.com> wrote:
>>>>>>
>>>>>>> Hi Matt, John,
>>>>>>>
>>>>>>> This is a great summary!
>>>>>>>
>>>>>>> >> The only reference I could find for VPP in Azure was in this link
>>>>>>> >> but it is very outdated:
>>>>>>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html
>>>>>>>
>>>>>>> > Yes, that doc is way outdated. I advise against following the
>>>>>>> instructions
>>>>>>> > in it.
>>>>>>>
>>>>>>> Matt or John, would you mind proposing a patch for that? Even a
>>>>>>> quick & dirty "here are the steps you need to follow" would be great.
>>>>>>>
>>>>>>> Best
>>>>>>> ben
>>>>>>>
>>>>>>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21347): https://lists.fd.io/g/vpp-dev/message/21347
Mute This Topic: https://lists.fd.io/mt/90759128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
