Package: linux-image-4.9.0-6-amd64
Version: 4.9.0-6-amd64
Debian Linux kernel 4.9.0-6-amd64 automatically spreads NVMe irq affinity among
all the CPUs, /proc/irq/n/smp_affinity looks good, but all the softirq still
goes to CPU 0 when running IO stress tests. Moreover because
IRQD_AFFINITY_MANAGED flag is set, manually writing to /proc/irq/n/smp_affinity
will fail like the follows:
# echo 2 > /proc/irq/70/smp_affinity
-bash: echo: write error: Input/output error
I used systemtap to force writing smp_affinity for NVMe irqs, here is what I
did:
# stap -g -e 'probe kernel.function("irq_can_set_affinity_usr").return {
$return = 1 }'
Then I manually wrote to each of the NVMe's smp_affinity, no errors anymore.
After this, the problem disappeared and the interrupts evenly distributed among
all the CPUs during IO stress tests.
I have been looking at the source code and hacked around, but couldn't pinpoint
the bug.
I believe this is a bug with 4.9 kernel, I tested upstream 4.9.98, it also had
the same problem. But the upstream kernel 4.16.7 had no problem, it worked
fine, unfortunately there are so many changes with irq code and nvme code
between 4.9 and 4.16.7, I couldn't figure out what code fixed the problem.
Wonder if someone can shed some lights here. Thanks in advance.
发自我的 iPhone
