> -----Original Message----- > From: Harish Patil [mailto:harish.patil at qlogic.com] > Sent: Monday, November 02, 2015 01:51 > To: Mussar, Gary; dev at dpdk.org > Subject: Re: [dpdk-dev] Status of bnx2x pmd? > > > > >I have been attempting to build testpmd with the bnx2x pmd enabled to try > >and talk through a bcm57810 10G NIC card. I have tried using SR-IOV VFs > >and I have tried using the PFs. Neither seem to be able to be properly > >initialized in testpmd and/or testpmd aborts. > > > >Is this PMD actually usable or is it trash? > > > >Gary > > > > Could you please explain the SR-IOV configuration that you are trying out? > Please provide all the details. > > Thanks,
I ultimately want to run dpdk in a VM with PCI pass through on SR-IOV ports. I have had no luck there so I thought I would start with DPDK on bare metal using the PFs. I am using Ubuntu 14.04.3 with the Ubuntu 3.19.0-31 kernel and dpdk from the dpdk.org master branch (d739ba4c6abf00eeeb680ab4e9540899182ea74d). # ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:04:00.0 'NetXtreme II BCM57810 10 Gigabit Ethernet' drv=igb_uio unused= 0000:04:00.1 'NetXtreme II BCM57810 10 Gigabit Ethernet' drv=igb_uio unused= Network devices using kernel driver =================================== 0000:05:00.0 'I350 Gigabit Network Connection' if=eth0 drv=igb unused=igb_uio *Active* 0000:05:00.1 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio Other network devices ===================== <none> Running testpmd: ./app/testpmd -c 7 -n 4 --socket-mem 900 -- -i --burst=64 --txqflags=0xf00 --nb-cores=2 --nb-ports=2 --portmask=0x3 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Detected lcore 8 as core 0 on socket 0 EAL: Detected lcore 9 as core 1 on socket 0 EAL: Detected lcore 10 as core 2 on socket 0 EAL: Detected lcore 11 as core 3 on socket 0 EAL: Detected lcore 12 as core 4 on socket 0 EAL: Detected lcore 13 as core 5 on socket 0 EAL: Detected lcore 14 as core 6 on socket 0 EAL: Detected lcore 15 as core 7 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 16 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up physically contiguous memory... EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffff4200000 (size = 0x200000) EAL: Ask a virtual area of 0x14a00000 bytes EAL: Virtual area found at 0x7fffdf600000 (size = 0x14a00000) EAL: Ask a virtual area of 0x56800000 bytes EAL: Virtual area found at 0x7fff88c00000 (size = 0x56800000) EAL: Ask a virtual area of 0x4000000 bytes EAL: Virtual area found at 0x7fff84a00000 (size = 0x4000000) EAL: Ask a virtual area of 0x54200000 bytes EAL: Virtual area found at 0x7fff30600000 (size = 0x54200000) EAL: Ask a virtual area of 0x14c00000 bytes EAL: Virtual area found at 0x7fff1b800000 (size = 0x14c00000) EAL: Ask a virtual area of 0x125400000 bytes EAL: Virtual area found at 0x7ffdf6200000 (size = 0x125400000) EAL: Ask a virtual area of 0x1b000000 bytes EAL: Virtual area found at 0x7ffddb000000 (size = 0x1b000000) EAL: Ask a virtual area of 0x18600000 bytes EAL: Virtual area found at 0x7ffdc2800000 (size = 0x18600000) EAL: Ask a virtual area of 0xf8600000 bytes EAL: Virtual area found at 0x7ffcca000000 (size = 0xf8600000) EAL: Ask a virtual area of 0x39400000 bytes EAL: Virtual area found at 0x7ffc90a00000 (size = 0x39400000) EAL: Ask a virtual area of 0x1fc00000 bytes EAL: Virtual area found at 0x7ffc70c00000 (size = 0x1fc00000) EAL: Ask a virtual area of 0x74c00000 bytes EAL: Virtual area found at 0x7ffbfbe00000 (size = 0x74c00000) EAL: Ask a virtual area of 0x9000000 bytes EAL: Virtual area found at 0x7ffbf2c00000 (size = 0x9000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffbf2800000 (size = 0x200000) EAL: Requesting 450 pages of size 2MB from socket 0 EAL: TSC frequency is ~1995380 KHz EAL: Master lcore 0 is ready (tid=f7fe4940;cpuset=[0]) PMD: rte_bnx2x_pmd_init(): >> PMD: rte_bnx2xvf_pmd_init(): >> EAL: lcore 2 is ready (tid=f5ec9700;cpuset=[2]) EAL: lcore 1 is ready (tid=f66ca700;cpuset=[1]) EAL: PCI device 0000:04:00.0 on NUMA socket 0 EAL: probe driver: 14e4:168e rte_bnx2x_pmd EAL: PCI memory mapped at 0x7ffff4400000 EAL: PCI memory mapped at 0x7ffff4c00000 EAL: PCI memory mapped at 0x7ffff5400000 PMD: eth_bnx2x_dev_init(): >> PMD: bnx2x_common_dev_init(): >> PMD: bnx2x_probe_pci_caps(): >> PMD: bnx2x_get_shmem_info(): >> PMD: bnx2x_acquire_hw_lock(): >> PMD: bnx2x_release_hw_lock(): >> PMD: bnx2x_get_phy_info(): >> PMD: bnx2x_reset_common(): >> EAL: PCI device 0000:04:00.1 on NUMA socket 0 EAL: probe driver: 14e4:168e rte_bnx2x_pmd EAL: PCI memory mapped at 0x7fffdee00000 EAL: PCI memory mapped at 0x7fffde600000 EAL: PCI memory mapped at 0x7ffff7fa1000 PMD: eth_bnx2x_dev_init(): >> PMD: bnx2x_common_dev_init(): >> PMD: bnx2x_probe_pci_caps(): >> PMD: bnx2x_get_shmem_info(): >> PMD: bnx2x_acquire_hw_lock(): >> PMD: bnx2x_release_hw_lock(): >> PMD: bnx2x_get_phy_info(): >> PMD: bnx2x_reset_common(): >> EAL: PCI device 0000:05:00.0 on NUMA socket 0 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:05:00.1 on NUMA socket 0 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped Interactive-mode selected Configuring Port 0 (socket 0) PMD: bnx2x_dev_configure(): >> PMD: bnx2x_dev_tx_queue_setup(): fp[00] req_bd=512, thresh=512, usable_bd=1020, total_bd=1024, tx_pages=4 PMD: bnx2x_dev_rx_queue_setup(): fp[00] req_bd=128, thresh=0, usable_bd=510, total_bd=512, rx_pages=1, cq_pages=8 PMD: bnx2x_dev_start(): >> PMD: bnx2x_nic_load(): >> PMD: bnx2x_ilt_set_info(): >> PMD: bnx2x_acquire_hw_lock(): >> PMD: bnx2x_set_pf_load(): >> PMD: bnx2x_release_hw_lock(): >> PMD: bnx2x_nic_load_request(): >> PMD: bnx2x_nic_load_pmf(): >> PMD: bnx2x_init_func_obj(): >> PMD: bnx2x_init_hw(): >> PMD: bnx2x_acquire_hw_lock(): >> PMD: bnx2x_reset_common(): >> PMD: bnx2x_release_hw_lock(): >> PANIC in bnx2x_write_dmae(): DMAE failed (-1)15: [./dpdk-2.1.0/app/testpmd() [0x438303]] 14: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7ffff6eedec5]] 13: [./dpdk-2.1.0/app/testpmd(main+0x2c1) [0x436a81]] 12: [./dpdk-2.1.0/app/testpmd(start_port+0x318) [0x43a8b8]] 11: [./dpdk-2.1.0/app/testpmd(rte_eth_dev_start+0xad) [0x573d2d]] 10: [./dpdk-2.1.0/app/testpmd() [0x466a24]] 9: [./dpdk-2.1.0/app/testpmd(bnx2x_init+0x21b) [0x476ffb]] 8: [./dpdk-2.1.0/app/testpmd() [0x471eb4]] 7: [./dpdk-2.1.0/app/testpmd(ecore_func_state_change+0x102) [0x4c4aa2]] 6: [./dpdk-2.1.0/app/testpmd() [0x4c1f62]] 5: [./dpdk-2.1.0/app/testpmd() [0x46ce47]] 4: [./dpdk-2.1.0/app/testpmd() [0x46c1e7]] 3: [./dpdk-2.1.0/app/testpmd() [0x468867]] 2: [./dpdk-2.1.0/app/testpmd(__rte_panic+0xc9) [0x430267]] 1: [./dpdk-2.1.0/app/testpmd(rte_dump_stack+0x1a) [0x4b3c3a]] ./startTestPmd: line 14: 19899 Aborted (core dumped) "$@" If I turn on more debugging, I can see: PMD: bnx2x_issue_dmae_with_comp(): DMAE timeout! PANIC in bnx2x_write_dmae(): DMAE failed (-1)16: [./dpdk-2.1.0/app/testpmd() [0x439043]] If running dpdk with the PFs on bare metal fails, I am not surprised that running with SR-IOV fails. # ./tools/dpdk_nic_bind.py --bind igb_uio 0000:04:01.0 0000:04:09.0 root at gary-desk:/home/skyfall# ifconfig eth4 up^C root at gary-desk:/home/skyfall# ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:04:01.0 'NetXtreme II BCM57810 10 Gigabit Ethernet Virtual Function' drv=igb_uio unused= 0000:04:09.0 'NetXtreme II BCM57810 10 Gigabit Ethernet Virtual Function' drv=igb_uio unused= Network devices using kernel driver =================================== 0000:04:00.0 'NetXtreme II BCM57810 10 Gigabit Ethernet' if=eth4 drv=bnx2x unused=igb_uio 0000:04:00.1 'NetXtreme II BCM57810 10 Gigabit Ethernet' if=eth5 drv=bnx2x unused=igb_uio 0000:05:00.0 'I350 Gigabit Network Connection' if=eth0 drv=igb unused=igb_uio *Active* 0000:05:00.1 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio Other network devices ===================== <none> Running testpmd with SR-IOV on bare metal: # ./app/testpmd -c 7 -n 4 --socket-mem 900 -- -i --burst=64 --txqflags=0xf00 --nb-cores=2 --nb-ports=2 --portmask=0x3 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Detected lcore 8 as core 0 on socket 0 EAL: Detected lcore 9 as core 1 on socket 0 EAL: Detected lcore 10 as core 2 on socket 0 EAL: Detected lcore 11 as core 3 on socket 0 EAL: Detected lcore 12 as core 4 on socket 0 EAL: Detected lcore 13 as core 5 on socket 0 EAL: Detected lcore 14 as core 6 on socket 0 EAL: Detected lcore 15 as core 7 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 16 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up physically contiguous memory... EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffff4200000 (size = 0x200000) EAL: Ask a virtual area of 0x14a00000 bytes EAL: Virtual area found at 0x7fffdf600000 (size = 0x14a00000) EAL: Ask a virtual area of 0x56800000 bytes EAL: Virtual area found at 0x7fff88c00000 (size = 0x56800000) EAL: Ask a virtual area of 0x4000000 bytes EAL: Virtual area found at 0x7fff84a00000 (size = 0x4000000) EAL: Ask a virtual area of 0x54200000 bytes EAL: Virtual area found at 0x7fff30600000 (size = 0x54200000) EAL: Ask a virtual area of 0x14c00000 bytes EAL: Virtual area found at 0x7fff1b800000 (size = 0x14c00000) EAL: Ask a virtual area of 0x125400000 bytes EAL: Virtual area found at 0x7ffdf6200000 (size = 0x125400000) EAL: Ask a virtual area of 0x1b000000 bytes EAL: Virtual area found at 0x7ffddb000000 (size = 0x1b000000) EAL: Ask a virtual area of 0x18600000 bytes EAL: Virtual area found at 0x7ffdc2800000 (size = 0x18600000) EAL: Ask a virtual area of 0xf8600000 bytes EAL: Virtual area found at 0x7ffcca000000 (size = 0xf8600000) EAL: Ask a virtual area of 0x39400000 bytes EAL: Virtual area found at 0x7ffc90a00000 (size = 0x39400000) EAL: Ask a virtual area of 0x1fc00000 bytes EAL: Virtual area found at 0x7ffc70c00000 (size = 0x1fc00000) EAL: Ask a virtual area of 0x74c00000 bytes EAL: Virtual area found at 0x7ffbfbe00000 (size = 0x74c00000) EAL: Ask a virtual area of 0x9000000 bytes EAL: Virtual area found at 0x7ffbf2c00000 (size = 0x9000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffbf2800000 (size = 0x200000) EAL: Requesting 450 pages of size 2MB from socket 0 EAL: TSC frequency is ~1995380 KHz EAL: Master lcore 0 is ready (tid=f7fe4940;cpuset=[0]) PMD: rte_bnx2x_pmd_init(): >> PMD: rte_bnx2xvf_pmd_init(): >> EAL: lcore 2 is ready (tid=f5ec9700;cpuset=[2]) EAL: lcore 1 is ready (tid=f66ca700;cpuset=[1]) EAL: PCI device 0000:04:00.0 on NUMA socket 0 EAL: probe driver: 14e4:168e rte_bnx2x_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:04:00.1 on NUMA socket 0 EAL: probe driver: 14e4:168e rte_bnx2x_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:04:01.0 on NUMA socket 0 EAL: probe driver: 14e4:16af rte_bnx2xvf_pmd EAL: PCI memory mapped at 0x7ffff4400000 EAL: PCI memory mapped at 0x7ffff4408000 PMD: eth_bnx2xvf_dev_init(): >> PMD: bnx2x_common_dev_init(): >> PMD: bnx2x_probe_pci_caps(): >> EAL: PCI device 0000:04:09.0 on NUMA socket 0 EAL: probe driver: 14e4:16af rte_bnx2xvf_pmd EAL: PCI memory mapped at 0x7ffff440a000 EAL: PCI memory mapped at 0x7ffff4412000 PMD: eth_bnx2xvf_dev_init(): >> PMD: bnx2x_common_dev_init(): >> PMD: bnx2x_probe_pci_caps(): >> Interactive-mode selected Configuring Port 0 (socket 0) PMD: bnx2x_dev_configure(): >> PMD: bnx2x_dev_tx_queue_setup(): fp[00] req_bd=512, thresh=512, usable_bd=1020, total_bd=1024, tx_pages=4 PMD: bnx2x_dev_rx_queue_setup(): fp[00] req_bd=128, thresh=0, usable_bd=510, total_bd=512, rx_pages=1, cq_pages=8 PMD: bnx2x_dev_start(): >> PMD: bnx2x_nic_load(): >> Fail to start port 0 Configuring Port 1 (socket 0) PMD: bnx2x_dev_configure(): >> PMD: bnx2x_dev_tx_queue_setup(): fp[00] req_bd=512, thresh=512, usable_bd=1020, total_bd=1024, tx_pages=4 PMD: bnx2x_dev_rx_queue_setup(): fp[00] req_bd=128, thresh=0, usable_bd=510, total_bd=512, rx_pages=1, cq_pages=8 PMD: bnx2x_dev_start(): >> PMD: bnx2x_nic_load(): >> Fail to start port 1 Please stop the ports first Done PMD: bnx2x_promisc_enable(): >> PMD: bnx2x_promisc_enable(): >> testpmd> Gary > Harish > > > ________________________________ > > This message and any attached documents contain information from the > sending company or its parent company(s), subsidiaries, divisions or > branch offices that may be confidential. If you are not the intended > recipient, you may not read, copy, distribute, or use this information. If > you have received this transmission in error, please notify the sender > immediately by reply e-mail and then delete this message.
