Verification successful on Ubuntu 14.04 (ESM):
No interrupts are generated on the PF by the link status check of the VF.
Verified with both Link down and Link up.
# uname -rv
3.13.0-174-generic #225-Ubuntu SMP Wed Oct 2 22:08:03 UTC 2019
# ls -ld /sys/bus/pci/drivers/ixgbe/*/net/*
drwxr-xr-x 5 root root 0 Oct 17 20:53
/sys/bus/pci/drivers/ixgbe/0000:04:00.0/net/eth4
drwxr-xr-x 5 root root 0 Oct 17 20:53
/sys/bus/pci/drivers/ixgbe/0000:04:00.1/net/eth5
# echo 1 > /sys/class/net/eth4/device/sriov_numvfs
# dmesg
...
[ 1104.000023] ixgbe 0000:04:00.0 eth4: SR-IOV enabled with 1 VFs
[ 1104.100390] pci 0000:04:10.0: [8086:1515] type 00 class 0x020000
[ 1104.100981] ixgbe 0000:04:00.0: removed PHC on eth4
[ 1104.109683] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function
Network Driver - version 2.11.3-k
[ 1104.109689] ixgbevf: Copyright (c) 2009 - 2012 Intel Corporation.
[ 1104.109797] ixgbevf 0000:04:10.0: enabling device (0000 -> 0002)
[ 1104.109976] ixgbe 0000:04:00.0 eth4: VF Reset msg received from vf 0
[ 1104.110096] ixgbe 0000:04:00.0: VF 0 has no MAC address assigned, you may
have to assign one manually
[ 1104.120353] ixgbevf 0000:04:10.0: PF still in reset state. Is the PF
interface up?
[ 1104.120359] ixgbevf 0000:04:10.0: Assigning random MAC address
[ 1104.121016] ixgbevf 0000:04:10.0: irq 305 for MSI/MSI-X
[ 1104.121037] ixgbevf 0000:04:10.0: irq 306 for MSI/MSI-X
[ 1104.516319] ixgbe 0000:04:00.0: irq 155 for MSI/MSI-X
[ 1104.516335] ixgbe 0000:04:00.0: irq 156 for MSI/MSI-X
[ 1104.516359] ixgbe 0000:04:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx
Queue count = 1
[ 1104.523447] pps pps1: new PPS source ptp4
[ 1104.523452] ixgbe 0000:04:00.0: registered PHC device on eth4
[ 1111.947668] ixgbe 0000:04:00.0 eth4: NIC Link is Up 10 Gbps, Flow Control:
None
# ls -ld /sys/bus/pci/drivers/ixgbevf/*/net/*
drwxr-xr-x 5 root root 0 Oct 17 21:11
/sys/bus/pci/drivers/ixgbevf/0000:04:10.0/net/enp4s16
Link down:
# ethtool enp4s16 | grep Link
Link detected: no
# while true; do ethtool enp4s16 >/dev/null; done
On another terminal:
$ while sleep 1; do echo -n "$(date) - "; cat /proc/interrupts | grep eth4 |
grep -v Tx | tr -s ' '; done
Thu Oct 17 21:21:08 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:09 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:10 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:11 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:12 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:13 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:14 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:15 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:16 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:17 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:18 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:19 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:20 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:21 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:22 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:23 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:24 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:25 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:26 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:27 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:28 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:21:29 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
^C
Link up:
# ip link set enp4s16 up
# ethtool enp4s16 | grep Link
Link detected: yes
# while true; do ethtool enp4s16 >/dev/null; done
On another terminal:
$ while sleep 1; do echo -n "$(date) - "; cat /proc/interrupts | grep eth4 |
grep -v Tx | tr -s ' '; done
Thu Oct 17 21:22:56 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:22:57 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:22:58 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:22:59 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:00 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:01 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:02 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:03 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:04 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:05 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:06 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:07 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:08 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:09 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:10 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:11 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:12 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
Thu Oct 17 21:23:13 UTC 2019 - 156: 3 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IR-PCI-MSI-edge eth4
^C
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1836760
Title:
ixgbe{vf} - Physical Function gets IRQ when VF checks link state
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Trusty:
Fix Committed
Status in linux source package in Xenial:
Fix Released
Status in linux source package in Bionic:
Fix Released
Status in linux source package in Cosmic:
Won't Fix
Status in linux source package in Disco:
Fix Released
Status in linux source package in Eoan:
Fix Released
Status in linux source package in Focal:
Fix Released
Bug description:
[Impact]
* Intel NICs that are SR-IOV capable and are managed by ixgbe driver presents
a potentially harmful behavior when the ixgbevf-managed VFs (Virtual Functions)
perform an ethtool link check. The ixgbevf driver issues a mailbox command in
the ethtool link state handler, which induces one IRQ in the PF (Physical
Function) per link check.
* This was reported as a sort of "denial-of-service" from a guest; due to
some link check loop running inside a guest with PCI-PT of a ixgbevf-managed
VF, the host received a huge amount of IRQs causing soft-lockups.
* The patch proposed in this SRU request fix this behavior by relying in the
saved link state (obtained in the ixgbevf's watchdog routine) instead of
issuing a mailbox command to the PF in every link state check request. The
commit is available on Linus tree: 1e1b0c658d9b ("ixgbevf: Use cached link
state instead of re-reading the value for ethtool")
http://git.kernel.org/linus/1e1b0c658d9b
[Test case]
Reproducing the behavior is pretty simple; having a machine with an
Intel NIC managed by ixgbe, proceed with the following steps:
a) Create one or more VFs
(echo 1 > /sys/class/net/<PF iface>/device/sriov_numvfs)
b) In a different terminal, monitor the non-TxRx PF IRQs:
(watch -n1 "cat /proc/interrupts | grep <PF iface> | grep -v Tx")
c) Run "ethtool <VF iface>" in a loop
Without the hereby proposed patch, the PF IRQs will increase.
[Regression potential]
The patch scope is restricted to ixgbevf ethtool link-check procedure,
and was developed by the vendor itself. Being a self-contained patch
affecting only this driver's ethtool handler, the worst potential
regression would be a wrong link state report.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1836760/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
