On Thu, 7 Sep 2017, Yu Chen wrote:
> On Wed, Sep 06, 2017 at 10:03:58AM +0200, Thomas Gleixner wrote:
> > Can you please apply the debug patch below, boot the machine and right
> > after login provide the output of
> >
> > # cat /sys/kernel/debug/tracing/trace
> >
> kworker/0:2-303 [000] .... 9.135467: msi_domain_alloc_irqs: dev:
> 0000:bb:00.0 nvec 1 virq 34
> kworker/0:2-303 [000] .... 9.135476: msi_domain_alloc_irqs: dev:
> 0000:bb:00.0 nvec 1 virq 35
> kworker/0:2-303 [000] .... 9.135484: msi_domain_alloc_irqs: dev:
> 0000:bb:00.0 nvec 1 virq 36
<SNIP>
> kworker/0:2-303 [000] .... 9.762268: msi_domain_alloc_irqs: dev:
> 0000:bb:00.3 nvec 1 virq 331
> kworker/0:2-303 [000] .... 9.762278: msi_domain_alloc_irqs: dev:
> 0000:bb:00.3 nvec 1 virq 332
> kworker/0:2-303 [000] .... 9.762288: msi_domain_alloc_irqs: dev:
> 0000:bb:00.3 nvec 1 virq 333
That's 300 vectors.
> bb:00.[0-3] Ethernet controller: Intel Corporation Device 37d0 (rev 03)
>
> -+-[0000:b2]-+-00.0-[b3-bc]----00.0-[b4-bc]--+-00.0-[b5-b6]----00.0
> | | +-01.0-[b7-b8]----00.0
> | | +-02.0-[b9-ba]----00.0
> | | \-03.0-[bb-bc]--+-00.0
> | | +-00.1
> | | +-00.2
> | | \-00.3
>
> and they are using i40e driver, the vectors should be reserved by:
> i40e_probe() ->
> i40e_init_interrupt_scheme() ->
> i40e_init_msix() ->
> i40e_reserve_msix_vectors() ->
> pci_enable_msix_range()
>
> # ls /sys/kernel/debug/irq/irqs
> 0 10 11 13 142 184 217 259 292 31 33
> 337 339 340 342 344 346 348 350 352 354 356
> 358 360 362 364 366 368 370 372 374 376 378
> 380 382 384 386 388 390 392 394 4 6 7 9
> 1 109 12 14 15 2 24 26 3 32 335
> 338 34 341 343 345 347 349 351 353 355 357
> 359 361 363 365 367 369 371 373 375 377 379
> 381 383 385 387 389 391 393 395 5 67 8
Out of these 300 interrupts exactly 8 randomly selected ones are actively
used. And the other 292 interrupts are just there because it might need
them in the future when the 32 CPU machine gets magically upgraded to 4096
cores at runtime?
Can the i40e people @intel please fix this waste of resources and sanitize
their interrupt allocation scheme?
Please switch it over to managed interrupts so the affinity spreading
happens in a sane way and the interrupts are properly managed on CPU
hotplug.
Thanks,
tglx
