Hello Helin, Konstantin, I noticed this issue quite some time ago (maybe around dpdk-1.3.0, not sure) but had no time to investigate/report. I hit it again with dpdk-2.2.0-rc2, so maybe all dpdk versions are impacted.
The test is quite simple : - send icmp packets from a system (tn) to itself using two 10G interfaces connected to two VF ports of a dual port 82599 nic, - run testpmd in a vm owning those two VF ports, - reset a pf interface in the host, - then no connectivity on the associated VF port unless it is completely restarted. In my previous test, I think I also unplugged the fiber then replugged (without touching the pf interface status on the host) and observed the same issue. I looked at ixgbevf pmd, but I can't find a place where the PF asking for reset would be handled except at devinit. Here is the full description of my test : The system owning the two VF ports is a ubuntu 14.04 virtual machine (3.13.0-71-generic kernel). The host running this virtual machine is running ubuntu 14.04 as well (but kernel is newer 3.19.0-39-generic). - host setup : echo 3072 >/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages echo 3072 >/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages mkdir -p /mnt/huge mount -t hugetlbfs none /mnt/huge echo 1 > "/sys/bus/pci/devices/0000:83:00.0/sriov_numvfs" bridge link set dev ixgbe0 hwmode veb ip link set dev ixgbe0 vf 0 mac 00:09:c0:0e:4e:64 ip link set dev ixgbe0 vf 0 spoofchk off ip link set dev ixgbe0 mtu 9000 ip link set dev ixgbe0 up ip link set dev ixgbe0 address 00:09:c0:0e:4e:64 echo 1 > "/sys/bus/pci/devices/0000:83:00.1/sriov_numvfs" bridge link set dev ixgbe1 hwmode veb ip link set dev ixgbe1 vf 0 mac 00:09:c0:0e:4e:65 ip link set dev ixgbe1 vf 0 spoofchk off ip link set dev ixgbe1 mtu 9000 ip link set dev ixgbe1 up ip link set dev ixgbe1 address 00:09:c0:0e:4e:65 qemu-system-x86_64 -k fr -daemonize -S --enable-kvm -m 3G -mem-path /mnt/huge -cpu host -smp cores=6,threads=2,sockets=1 -serial telnet::53714,server,nowait -serial null -monitor telnet::36576,server,nowait -device virtio-net,mac=DE:AD:DE:01:02:03,netdev=user.0,addr=03 -netdev user,id=user.0 -device pci-assign,host=0000:83:10.0,addr=04 -device pci-assign,host=0000:83:10.1,addr=05 -hda ubuntu-14.04-template.qcow2 -snapshot -vga none -display none - guest setup : echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages mkdir -p /mnt/huge mount -t hugetlbfs none /mnt/huge modprobe uio modprobe igb_uio echo 0000:00:04.0 > /sys/m/bus/pci/drivers/ixgbevf/unbind echo 0000:00:05.0 > /sys/m/bus/pci/drivers/ixgbevf/unbind echo 8086 10ed > /sys/bus/pci/drivers/igb_uio/new_id testpmd -w0000:00:04.0 -w0000:00:05.0 --huge-dir=/mnt/huge -n 4 -c 0xe -- -i [snip] Port 0: 00:09:C0:0E:4E:64 [snip] Port 1: 00:09:C0:0E:4E:65 Checking link statuses... Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed 10000 Mbps - full-duplex Done testpmd> start - tn setup : ip link set dev xaui0 up ip link set dev xaui1 up ip addr add 1.1.1.1/24 dev xaui0 arp -i xaui0 -s 1.1.1.2 00:09:c0:0e:4e:64 - From here, on tn, send icmp packets with a tcpdump running background on the other iface : ping 1.1.1.2 PING 1.1.1.2 (1.1.1.2): 56 data bytes 03:10:15.003047 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 1 03:10:16.003151 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 2 03:10:17.003246 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 3 - Ok, now, kick ixgbe0 pf interface down / up on host system. ip link set dev ixgbe0 down ip link set dev ixgbe0 up - On tn, no packet is received anymore, the only workaround I found is to stop and restart the port 0 : testpmd> stop [snip] Done. testpmd> port stop all Stopping ports... Checking link statuses... Port 0 Link Down Port 1 Link Up - speed 10000 Mbps - full-duplex Done Funny thing here, the link is reported down, while the pf is already back up. testpmd> port start all PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0). Port 0: 00:09:C0:0E:4E:64 PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=1). Port 1: 00:09:C0:0E:4E:65 Checking link statuses... Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed 10000 Mbps - full-duplex Done Thanks. -- David Marchand
