Hello, I was trying to check "vanilla" VPP, without integration of patches from NXP: it works well on LX2160A board with PCIE NICs (I'm using IGBs), but fails to start with DPAA2 ports enabled (NXP-patched version starts with DPAA2, but very unstable with both DPAA2 and PCIE ethernets).
"Vanilla" VPP was built on board natively, by simple VPP build instructions, either directly on in vagrant. Different versions were checked: - 21.06 - 22.10 - 23.02 RC as-is, and with patch to bump DPDK 22.11, all are with same results, PCIE NICs only configuration works well and stable, crashes on initialization of DPAA2 devices. So, when DPAA2 ports are enabled, and DPRC=dprc.X pointing to container with interfaces is set, we see following on start of standard vanilla VPP (traces are identical for versions above). Any ideas where to look to fix an issue? (gdb) run -c /etc/vpp/startup.conf Starting program: /home/abramov/vpp-stable/build-root/install-vpp_debug-native/vpp/bin/vpp -c /etc/vpp/startup.conf [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1". [New Thread 0xffff716e6140 (LWP 1679)] [New Thread 0xffff70ee5140 (LWP 1680)] [New Thread 0xffff6bfff140 (LWP 1681)] Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. dpaa2_dev_rx_queue_setup (dev=0xffff765c5c80 <rte_eth_devices>, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db3c0, mb_pool=0x170cac180) at ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c:675 675 ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c: No such file or directory. (gdb) bt #0 dpaa2_dev_rx_queue_setup (dev=0xffff765c5c80 <rte_eth_devices>, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db3c0, mb_pool=0x170cac180) at ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c:675 #1 0x0000ffff74eaf518 in rte_eth_rx_queue_setup (port_id=0, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db478, mp=0x170cac180) at ../src-dpdk/lib/librte_ethdev/rte_ethdev.c:2115 #2 0x0000ffff7596b020 in dpdk_device_setup (xd=0xffff7c171500) at /root/vpp-stable2/src/plugins/dpdk/device/common.c:133 #3 0x0000ffff7598b824 in dpdk_lib_init (dm=0xffff765b1220 <dpdk_main>) at /root/vpp-stable2/src/plugins/dpdk/device/init.c:805 #4 0x0000ffff75989874 in dpdk_process (vm=0xffff76c00680, rt=0xffff7a9da280, f=0x0) at /root/vpp-stable2/src/plugins/dpdk/device/init.c:1840 #5 0x0000fffff727aaf4 in vlib_process_bootstrap (_a=281472624744504) at /root/vpp-stable2/src/vlib/main.c:1284 #6 0x0000fffff7121348 in clib_calljmp () at /root/vpp-stable2/src/vppinfra/longjmp.S:809 Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) (gdb) run -c /etc/vpp/startup.conf Starting program: /home/abramov/vpp-stable/build-root/install-vpp_debug-native/vpp/bin/vpp -c /etc/vpp/startup.conf [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1". [New Thread 0xffff716e6140 (LWP 15156)] [New Thread 0xffff70ee5140 (LWP 15157)] [New Thread 0xffff6bfff140 (LWP 15158)] Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. dpaa2_dev_rx_queue_setup (dev=0xffff765c5c80 <rte_eth_devices>, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db3c0, mb_pool=0x170cac180) at ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c:675 675 ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c: No such file or directory. (gdb) bt full #0 dpaa2_dev_rx_queue_setup (dev=0xffff765c5c80 <rte_eth_devices>, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db3c0, mb_pool=0x170cac180) at ../src-dpdk/drivers/net/dpaa2/dpaa2_ethdev.c:675 priv = 0x2184295180 dpni = 0x2184295080 dpaa2_q = 0xffff719db3f8 cfg = {destination = {id = 49536, type = DPNI_DEST_DPIO, hold_active = 0 '\000', priority = 179 '\263'}, user_context = 281472643297432, flc = {value = 5035480, stash_control = 128 '\200'}, cgid = 1024} options = 0 '\000' flow_id = 113 'q' bpid = 65535 i = 65535 ret = 1961568588 __func__ = "dpaa2_dev_rx_queue_setup" #1 0x0000ffff74eaf518 in rte_eth_rx_queue_setup (port_id=0, rx_queue_id=0, nb_rx_desc=1024, socket_id=0, rx_conf=0xffff719db478, mp=0x170cac180) at ../src-dpdk/lib/librte_ethdev/rte_ethdev.c:2115 ret = 0 mbp_buf_size = 2176 dev = 0xffff765c5c80 <rte_eth_devices> dev_info = {device = 0x7edc90, driver_name = 0xffff75a2c2a8 "net_dpaa2", if_index = 0, min_mtu = 68, max_mtu = 65535, dev_flags = 0x2184297e9c, min_rx_bufsize = 512, max_rx_pktlen = 10240, max_lro_pkt_size = 0, max_rx_queues = 128, max_tx_queues = 16, max_mac_addrs = 16, max_hash_mac_addrs = 0, max_vfs = 0, max_vmdq_pools = 16, rx_seg_capa = {multi_pools = 0, offset_allowed = 0, offset_align_log2 = 0, max_nseg = 0, reserved = 0}, rx_offload_capa = 944719, tx_offload_capa = 114847, rx_queue_offload_capa = 0, tx_queue_offload_capa = 0, reta_size = 0, hash_key_size = 0 '\000', flow_type_rss_offloads = 8590196732, default_rxconf = {rx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, rx_free_thresh = 0, rx_drop_en = 0 '\000', rx_deferred_start = 0 '\000', rx_nseg = 0, offloads = 0, rx_seg = 0x0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, default_txconf = {tx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, tx_rs_thresh = 0, tx_free_thresh = 0, tx_deferred_start = 0 '\000', offloads = 0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, vmdq_queue_base = 0, vmdq_queue_num = 0, vmdq_pool_base = 0, rx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, tx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, speed_capa = 23904, nb_rx_queues = 1, nb_tx_queues = 3, default_rxportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, default_txportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, dev_capa = 0, switch_info = {name = 0x0, domain_id = 65535, port_id = 0}, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}} local_conf = {rx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, rx_free_thresh = 0, rx_drop_en = 0 '\000', rx_deferred_start = 0 '\000', rx_nseg = 0, offloads = 0, rx_seg = 0x0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}} rxq = 0x170cabd80 __func__ = "rte_eth_rx_queue_setup" #2 0x0000ffff7596b020 in dpdk_device_setup (xd=0xffff7c1e2880) at /root/vpp-stable2/src/plugins/dpdk/device/common.c:133 rxq = 0xffff7c160b00 bpidx = 0 '\000' bp = 0xffff7a98b300 mp = 0x170cac180 dm = 0xffff765b1220 <dpdk_main> --Type <RET> for more, q to quit, c to continue without paging-- vm = 0xffff76c00680 vnm = 0xfffff7f5f2f8 <vnet_main> tm = 0xfffff73194e0 <vlib_thread_main> sw = 0xffff7c1786cc hi = 0xffff7c137680 dev_info = {device = 0x7edc90, driver_name = 0xffff75a2c2a8 "net_dpaa2", if_index = 0, min_mtu = 68, max_mtu = 65535, dev_flags = 0x2184297e9c, min_rx_bufsize = 512, max_rx_pktlen = 10240, max_lro_pkt_size = 0, max_rx_queues = 128, max_tx_queues = 16, max_mac_addrs = 16, max_hash_mac_addrs = 0, max_vfs = 0, max_vmdq_pools = 16, rx_seg_capa = {multi_pools = 0, offset_allowed = 0, offset_align_log2 = 0, max_nseg = 0, reserved = 0}, rx_offload_capa = 944719, tx_offload_capa = 114847, rx_queue_offload_capa = 0, tx_queue_offload_capa = 0, reta_size = 0, hash_key_size = 0 '\000', flow_type_rss_offloads = 8590196732, default_rxconf = {rx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, rx_free_thresh = 0, rx_drop_en = 0 '\000', rx_deferred_start = 0 '\000', rx_nseg = 0, offloads = 0, rx_seg = 0x0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, default_txconf = {tx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, tx_rs_thresh = 0, tx_free_thresh = 0, tx_deferred_start = 0 '\000', offloads = 0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, vmdq_queue_base = 0, vmdq_queue_num = 0, vmdq_pool_base = 0, rx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, tx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, speed_capa = 23904, nb_rx_queues = 0, nb_tx_queues = 0, default_rxportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, default_txportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, dev_capa = 0, switch_info = {name = 0x0, domain_id = 65535, port_id = 0}, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}} bitmap = 0 rv = 0 j = 0 #3 0x0000ffff7598b824 in dpdk_lib_init (dm=0xffff765b1220 <dpdk_main>) at /root/vpp-stable2/src/plugins/dpdk/device/init.c:805 dev_info = {device = 0x7edc90, driver_name = 0xffff75a2c2a8 "net_dpaa2", if_index = 0, min_mtu = 68, max_mtu = 65535, dev_flags = 0x2184297e9c, min_rx_bufsize = 512, max_rx_pktlen = 10240, max_lro_pkt_size = 0, max_rx_queues = 128, max_tx_queues = 16, max_mac_addrs = 16, max_hash_mac_addrs = 0, max_vfs = 0, max_vmdq_pools = 16, rx_seg_capa = {multi_pools = 0, offset_allowed = 0, offset_align_log2 = 0, max_nseg = 0, reserved = 0}, rx_offload_capa = 944719, tx_offload_capa = 114847, rx_queue_offload_capa = 0, tx_queue_offload_capa = 0, reta_size = 0, hash_key_size = 0 '\000', flow_type_rss_offloads = 8590196732, default_rxconf = {rx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, rx_free_thresh = 0, rx_drop_en = 0 '\000', rx_deferred_start = 0 '\000', rx_nseg = 0, offloads = 0, rx_seg = 0x0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, default_txconf = {tx_thresh = {pthresh = 0 '\000', hthresh = 0 '\000', wthresh = 0 '\000'}, tx_rs_thresh = 0, tx_free_thresh = 0, tx_deferred_start = 0 '\000', offloads = 0, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, vmdq_queue_base = 0, vmdq_queue_num = 0, vmdq_pool_base = 0, rx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, tx_desc_lim = {nb_max = 65535, nb_min = 0, nb_align = 1, nb_seg_max = 65535, nb_mtu_seg_max = 65535}, speed_capa = 23904, nb_rx_queues = 0, nb_tx_queues = 0, default_rxportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, default_txportconf = {burst_size = 32, ring_size = 512, nb_queues = 1}, dev_capa = 0, switch_info = {name = 0x0, domain_id = 65535, port_id = 0}, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}} next_port_id = 0 devconf = 0xffff765b12a0 <dpdk_config_main+48> p = 0x0 q = 1 pci_dev = 0x0 vmbus_dev = 0x0 pci_addr = {{domain = 0, bus = 0 '\000', slot = 0 '\000', function = 0 '\000'}, as_u32 = 0} --Type <RET> for more, q to quit, c to continue without paging-- vmbus_addr = {guid = '\000' <repeats 15 times>, as_u32 = {0, 0, 0, 0}} addr = "\320c\264\003\034\312" vlan_off = 0 nports = 3 mtu = 9216 max_rx_frame = 9220 i = 0 error = 0x0 vm = 0xffff76c00680 tm = 0xfffff73194e0 <vlib_thread_main> vdm = 0xfffff7f5f038 <vnet_device_main> sw = 0xffff7c1786cc hi = 0xffff7c137680 xd = 0xffff7c1e2880 last_pci_addr = {{domain = 65535, bus = 255 '\377', slot = 31 '\037', function = 7 '\a'}, as_u32 = 4294967295} last_pci_addr_port = 0 af_packet_instance_num = 0 '\000' #4 0x0000ffff75989874 in dpdk_process (vm=0xffff76c00680, rt=0xffff7a9da280, f=0x0) at /root/vpp-stable2/src/plugins/dpdk/device/init.c:1840 error = 0x0 dm = 0xffff765b1220 <dpdk_main> xd = 0x0 tm = 0xfffff73194e0 <vlib_thread_main> now = 0 #5 0x0000fffff727aaf4 in vlib_process_bootstrap (_a=281472624744504) at /root/vpp-stable2/src/vlib/main.c:1284 a = 0xffff73cfd838 vm = 0xffff76c00680 node = 0xffff7a9da280 f = 0x0 p = 0xffff7a9da280 n = 0 #6 0x0000fffff7121348 in clib_calljmp () at /root/vpp-stable2/src/vppinfra/longjmp.S:809 No locals. Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) (gdb)
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22484): https://lists.fd.io/g/vpp-dev/message/22484 Mute This Topic: https://lists.fd.io/mt/96374294/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-