Hi, David We also noticed this issue before. And we are planning to fix this issue. And the patch for i40e is ready: http://dpdk.org/dev/patchwork/patch/9832/ http://dpdk.org/dev/patchwork/patch/9833/
The solution is that: PF reset interrupt will be captured by DPDK VF, then DPDK Call the application's callback function to reset the VF device. For i40e device, application need to detach and re-attach the device. For ixgbe, I think the solution would be similar. Thanks Jingjing > -----Original Message----- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of David Marchand > Sent: Friday, December 11, 2015 10:47 PM > To: Zhang, Helin; Ananyev, Konstantin > Cc: dev at dpdk.org > Subject: [dpdk-dev] ixgbevf does not recover from pf reset > > Hello Helin, Konstantin, > > I noticed this issue quite some time ago (maybe around dpdk-1.3.0, not > sure) but had no time to investigate/report. > I hit it again with dpdk-2.2.0-rc2, so maybe all dpdk versions are impacted. > > The test is quite simple : > - send icmp packets from a system (tn) to itself using two 10G interfaces > connected to two VF ports of a dual port 82599 nic, > - run testpmd in a vm owning those two VF ports, > - reset a pf interface in the host, > - then no connectivity on the associated VF port unless it is completely > restarted. > > In my previous test, I think I also unplugged the fiber then replugged > (without touching the pf interface status on the host) and observed the > same issue. > > I looked at ixgbevf pmd, but I can't find a place where the PF asking for > reset > would be handled except at devinit. > > > > Here is the full description of my test : > > The system owning the two VF ports is a ubuntu 14.04 virtual machine > (3.13.0-71-generic kernel). > The host running this virtual machine is running ubuntu 14.04 as well (but > kernel is newer 3.19.0-39-generic). > > > - host setup : > echo 3072 > >/sys/devices/system/node/node0/hugepages/hugepages- > 2048kB/nr_hugepages > echo 3072 > >/sys/devices/system/node/node1/hugepages/hugepages- > 2048kB/nr_hugepages > mkdir -p > /mnt/huge > mount -t hugetlbfs none /mnt/huge > > echo 1 > > "/sys/bus/pci/devices/0000:83:00.0/sriov_numvfs" > bridge link set dev ixgbe0 hwmode veb > ip link set dev ixgbe0 vf 0 mac 00:09:c0:0e:4e:64 ip link set dev ixgbe0 vf 0 > spoofchk off ip link set dev ixgbe0 mtu 9000 ip link set dev ixgbe0 up ip link > set dev ixgbe0 address 00:09:c0:0e:4e:64 > > echo 1 > > "/sys/bus/pci/devices/0000:83:00.1/sriov_numvfs" > bridge link set dev ixgbe1 hwmode veb > ip link set dev ixgbe1 vf 0 mac 00:09:c0:0e:4e:65 ip link set dev ixgbe1 vf 0 > spoofchk off ip link set dev ixgbe1 mtu 9000 ip link set dev ixgbe1 up ip link > set dev ixgbe1 address 00:09:c0:0e:4e:65 > > qemu-system-x86_64 -k fr -daemonize -S --enable-kvm -m 3G -mem-path > /mnt/huge -cpu host -smp cores=6,threads=2,sockets=1 -serial > telnet::53714,server,nowait -serial null -monitor telnet::36576,server,nowait > -device > virtio-net,mac=DE:AD:DE:01:02:03,netdev=user.0,addr=03 -netdev > user,id=user.0 -device pci-assign,host=0000:83:10.0,addr=04 -device > pci-assign,host=0000:83:10.1,addr=05 -hda ubuntu-14.04-template.qcow2 - > snapshot -vga none -display none > > > - guest setup : > echo 1024 > > /sys/devices/system/node/node0/hugepages/hugepages- > 2048kB/nr_hugepages > mkdir -p > /mnt/huge > mount -t hugetlbfs none /mnt/huge > > modprobe uio > modprobe igb_uio > echo 0000:00:04.0 > /sys/m/bus/pci/drivers/ixgbevf/unbind > echo 0000:00:05.0 > /sys/m/bus/pci/drivers/ixgbevf/unbind > echo 8086 10ed > /sys/bus/pci/drivers/igb_uio/new_id > > testpmd -w0000:00:04.0 -w0000:00:05.0 --huge-dir=/mnt/huge -n 4 -c 0xe -- -i > [snip] Port 0: > 00:09:C0:0E:4E:64 > [snip] > Port 1: > 00:09:C0:0E:4E:65 > Checking link > statuses... > Port 0 Link Up - speed 10000 Mbps - > full-duplex > Port 1 Link Up - speed 10000 Mbps - > full-duplex > Done > > testpmd> > start > > > - tn setup : > ip link set dev xaui0 up > ip link set dev xaui1 up > ip addr add 1.1.1.1/24 dev xaui0 > arp -i xaui0 -s 1.1.1.2 00:09:c0:0e:4e:64 > > > > - From here, on tn, send icmp packets with a tcpdump running background > on the other iface : > ping 1.1.1.2 > PING 1.1.1.2 (1.1.1.2): 56 data > bytes > 03:10:15.003047 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 > (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 1 > 03:10:16.003151 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 > (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 2 > 03:10:17.003246 00:02:bb:a8:0d:10 > 00:09:c0:0e:4e:64, ethertype IPv4 > (0x0800), length 98: IP 1.1.1.1 > 1.1.1.2: icmp 64: echo request seq 3 > > > - Ok, now, kick ixgbe0 pf interface down / up on host system. > ip link set dev ixgbe0 down > ip link set dev ixgbe0 up > > > - On tn, no packet is received anymore, the only workaround I found is to > stop and restart the port 0 : > > testpmd> > stop > [snip] > Done. > > testpmd> port stop > all > Stopping > ports... > Checking link > statuses... > Port 0 Link > Down > Port 1 Link Up - speed 10000 Mbps - > full-duplex > Done > > > Funny thing here, the link is reported down, while the pf is already back up. > > testpmd> port start all > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst > size no less than 4 (port=0). > Port 0: > 00:09:C0:0E:4E:64 > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst > size no less than 4 (port=1). > Port 1: > 00:09:C0:0E:4E:65 > Checking link > statuses... > Port 0 Link Up - speed 10000 Mbps - > full-duplex > Port 1 Link Up - speed 10000 Mbps - > full-duplex > Done > > > > Thanks. > > -- > David Marchand
