Hi all, I encountered a new behavior with ixgbe interrupt rates on 11-STABLE which was not present in 10-STABLE. We disable aim and manually set the interrupt rate. On 10-STABLE the interrupt rate will not exceed the configured value, but it is not the case in 11-STABLE. I am wondering if this is expected behavior or not.
Here is what I see:
# sysctl dev.ix | grep interrupt_rate
dev.ix.1.queue7.interrupt_rate: 500000
dev.ix.1.queue6.interrupt_rate: 500000
dev.ix.1.queue5.interrupt_rate: 100000
dev.ix.1.queue4.interrupt_rate: 500000
dev.ix.1.queue3.interrupt_rate: 500000
dev.ix.1.queue2.interrupt_rate: 500000
dev.ix.1.queue1.interrupt_rate: 500000
dev.ix.1.queue0.interrupt_rate: 500000
dev.ix.0.queue7.interrupt_rate: 500000
dev.ix.0.queue6.interrupt_rate: 500000
dev.ix.0.queue5.interrupt_rate: 100000
dev.ix.0.queue4.interrupt_rate: 5813
dev.ix.0.queue3.interrupt_rate: 500000
dev.ix.0.queue2.interrupt_rate: 500000
dev.ix.0.queue1.interrupt_rate: 500000
dev.ix.0.queue0.interrupt_rate: 500000
Our setup:
# sysctl hw.ix
hw.ix.rxd: 4096
hw.ix.txd: 4096
hw.ix.num_queues: 8
hw.ix.enable_msix: 1
hw.ix.advertise_speed: 0
hw.ix.flow_control: 0
hw.ix.tx_process_limit: -1
hw.ix.rx_process_limit: -1
hw.ix.max_interrupt_rate: 16000
hw.ix.enable_aim: 0
# sysctl dev.ix | grep aim
dev.ix.1.enable_aim: 0
dev.ix.0.enable_aim: 0
# vmstat -ai | grep ix
irq265: ix0:q0 1252479695 10058
irq266: ix0:q1 1232959515 9901
irq267: ix0:q2 1227101328 9854
irq268: ix0:q3 1229457979 9873
irq269: ix0:q4 1237070699 9934
irq270: ix0:q5 1231407684 9889
irq271: ix0:q6 1234032662 9910
irq272: ix0:q7 1230403197 9881
irq273: ix0:link 107 0
irq274: ix1:q0 2928503268 23517
irq275: ix1:q1 2526264197 20287
irq276: ix1:q2 2543700719 20427
irq277: ix1:q3 2552022026 20494
irq278: ix1:q4 2584441166 20754
irq279: ix1:q5 2563593862 20587
irq280: ix1:q6 2567670402 20619
irq281: ix1:q7 2581949876 20734
irq282: ix1:link 2 0
Hardware:
# pciconf -lv | grep -A3 ix
ix0@pci0:5:0:0: class=0x020000 card=0x7b118086 chip=0x154d8086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet 10G 2P X520 Adapter'
class = network
--
ix1@pci0:5:0:1: class=0x020000 card=0x7b118086 chip=0x154d8086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet 10G 2P X520 Adapter'
class = network
# dmesg | grep ^ix
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.13-k> port
0x2020-0x203f mem 0x92300000-0x923fffff,0x92404000-0x92407fff irq 34 at device
0.0 numa-domain 0 on pci5
ix0: Using MSIX interrupts with 9 vectors
ix0: ixgbe_allocate_msix: number of queues (8) != number of RSS buckets (128);
performance will be impacted.
ix0: Ethernet address: a0:36:9f:da:05:4c
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: Error 2 setting up SR-IOV
ix0: netmap queues/slots: TX 8/4096, RX 8/4096
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.1.13-k> port
0x2000-0x201f mem 0x92200000-0x922fffff,0x92400000-0x92403fff irq 37 at device
0.1 numa-domain 0 on pci5
ix1: Using MSIX interrupts with 9 vectors
ix1: ixgbe_allocate_msix: number of queues (8) != number of RSS buckets (128);
performance will be impacted.
ix1: Ethernet address: a0:36:9f:da:05:4e
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: Error 2 setting up SR-IOV
ix1: netmap queues/slots: TX 8/4096, RX 8/4096
Any ideas?
Thanks,
Babak
signature.asc
Description: OpenPGP digital signature
