schrieb Harald Schmalzbauer am 23.10.2012 11:49 (localtime): > schrieb Harald Schmalzbauer am 22.10.2012 21:48 (localtime): >> schrieb Harald Schmalzbauer am 22.10.2012 21:33 (localtime): >>> Hello, >>> >>> when using igb as module, no packet is received. >>> If I send out anything, I see the packet with tcpdump, also the switch >>> learns the MAC address, but nothing comes back in - total silenc, no >>> boradcasts, nothing. >>> If I unload the module and load it again, everything works as expected! >>> No matter if I load it by 4th loader, or later, I always have tio unload >>> first then load it again. >>> I'ts late here, I'll see tomorrow if things change when compieled into >>> kernel. > It doesn't matter if igb is loaded as module or compiled into kernel. > >>> Maby somebody has an idea what the source of the problem could be. >>> Please find atteched some info, the OS is 9-RC2-amd64 on ESXi5.1 and >>> nics are pci-passthrough. >> I found one possibly relevant difference: >> >> Non-Working state: dev.igb.0.link_irq: 0 >> Working state: dev.igb.0.link_irq: 2 > This is only true with msi-x!!! > If I disable mis-x, the problem itself vanishes. igb just works fine > from the initial loading (with dev.igb.0.link_irq=0!). > So dev.igb.0.link_irq is only relevant with msi-x. > But what makes me curious is why it also works mith mis-x enabled after > the second kldload!?! I think I found the root cause: When ESXi powers up the guest, the passthru-devices are intialized with: VMKPCIPassthru: 2565: BDF = 02:00.1 intrType = 2 numVectors: 1 intrType=2 seems to mean MSI. I guess, IOMMUIntel is instructed to remap one irq-vector for the device. But igb uses MSI-X and wants 3 vectors. If I unload if_igb and reload again, the ESXi-log shows the following: VMKPCIPassthru: 2565: BDF = 02:00.1 intrType = 4 numVectors: 3 intrType=4 seems to mean MSI-X. After that initialization, if_igb works fine and saves 25kIRQ/s!
I haven't found a way to change the power-up behaviour for the guest with ESXi. Is it possible to re-init a pci device from userland? Thanks, -Harry
signature.asc
Description: OpenPGP digital signature
