On 2017-08-22 19:38, Alex Williamson wrote:
On Mon, 21 Aug 2017 23:09:55 +0200
Torbjorn Jansson <torbjorn.jans...@mbox200.swipnet.se> wrote:
On 2017-08-21 19:56, Torbjorn Jansson wrote:
On 2017-08-19 07:37, Torbjorn Jansson wrote:
On 2017-08-19 02:34, taii...@gmx.com wrote:
On 08/15/2017 02:12 AM, Torbjorn Jansson wrote:
On 2017-08-15 02:44, taii...@gmx.com wrote:
On 08/14/2017 12:28 AM, Torbjorn Jansson wrote:
yes i noticed that part was missing, but the card i have is an I350-T2
and according to:
https://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005722.html
the I350-T2 is supposed to have sr-iov support.
i guess it is possible they sent me the wrong type of card or something,
i'll double check what is listed on the card itself.
Some OEM's (ex: dell) shut off the SR-IOV ability on their NIC's because
reasons.
Please provide an lspci -n and lspci -v -v for that device so we can see
what OEM it is.
I would return it and have them send you a real one that isn't nerfed,
also FYI most NIC's on ebay are counterfeit so be careful what you buy.
https://www.servethehome.com/investigating-fake-intel-i350-network-adapters/
(real intel NIC ASIC and SR-IOV but crappy secondary components)
if i look at the back of the card there is a sticker with: I350T2V2
here is the output:
Weird.
I would say you almost definitely got a fake - either way send it back.
yes very strange.
thing is, i bought it from a well known web shop I've been using forever.
i have opened a support ticket with them about this, hopefully i can return it.
but then question is, how do i find a network card with sr-iov that works?
clearly specs on web shops can't always be trusted.
also strange is that this page:
http://ark.intel.com/products/84804/Intel-Ethernet-Server-Adapter-I350-T2V2
clearly lists sr-iov and this is the model i got.
i did a bit more searching and checking.
found this pdf:
https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/ethernet-controller-i350-datasheet.pdf
section 6.2.25 looks interesting specifically the IOV Enable bit.
i used ethtool to dump the eeprom to file and a hex editor to check the
contents and this bit is set to a zero indicating IOV is not exposed.
but i'm definitely no expert on these things so i might be wrong too.
but i wonder if there is a way to flip this bit and see what happens.
i fixed it :)
i figured out how to use ethtool to flip the bit, rebooted and:
-------
03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection
(rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter I350-T2
Physical Slot: 6-1
Flags: bus master, fast devsel, latency 0, IRQ 61, NUMA node 0
Memory at fad00000 (32-bit, non-prefetchable) [size=1M]
Memory at fae04000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Capabilities: [a0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number a0-36-9f-ff-ff-ed-43-b0
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
Capabilities: [1a0] Transaction Processing Hints
Capabilities: [1c0] Latency Tolerance Reporting
Capabilities: [1d0] Access Control Services
Kernel driver in use: igb
Kernel modules: igb
-------
as you can see above sr-iov is now listed in lspci, needed sriov entries is now
listed under /sys and i can enable the virtual cards.
had to blacklist igbvf and libvirt kept complaining until i added the virtual
cards to vfio too.
so now i finally got it all working and have a vm with a virtual card.
but i think it is really stupid of the manufacture(s) (intel?) to disable
sr-iov in the eeprom and not provide any proper tool or documentation on how to
enable it.
Would you be so kind as to write-up instructions how you used ethtool
to resolve this so that others don't need to re-invent your solution?
Thanks,
yes of course.
first of all, be careful when you poke around in the cards eeprom.
below is what i did and it worked on my I350-T2V2 card.
theoretically it might work on other I350 cards too but it is not something i
have tested.
before starting you need to know the device name of your ethernet device, it is
also useful to check output of lspci -v on your device and check that your card
is missing this line:
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
first make a backup of your eeprom:
ethtool --eeprom-dump DEVNAME raw on >eth-eeprom.bin
i used this both as backup and to check the contents with a hex editor and to
make sure i understood the documentation correctly regarding endianess and offsets.
next we want to check what the iov enable bit is currently set to.
according to documentation i found, this is bit zero of word 0x25 in eeprom.
so this makes byte offset 0x4a (0x25*2 documentation uses 16 bit words so we
need to multiply by 2 to get byte offset)
so:
ethtool --eeprom-dump DEVNAME offset 0x4a length 1
in my case i got 0xf6 back.
so bit zero is set to a zero which means IOV capability is not exposed
(expected based on earlier lspci -v).
there is no point in continuing unless it is a zero.
take above value you just read out and set bit zero to 1 and then use your
newly calculated value in below command to change it:
ethtool --change-eeprom DEVNAME magic 0x15218086 offset 0x4a length 1 value
0xYOUR_VALUE_GOES_HERE
the magic value to ethtool is i think deliberately not documented to prevent
you from accidentally screwing up your eeprom.
in this case it is the pci vendor and device id (lspci -n).
once this is done, reboot computer and check output of lspci -v and look for:
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
if everything worked out it should now show up and you can go ahead with the
rest of sr-iov configuration.
again, be careful when you do this.
if you need to use this then i would recommend contacting intel support about
this because i think this is not the proper way of doing things.
i consider this more of a "hack" or a workaround than a proper solution.
i would rather have a proper tool or documentation or something from intel
specifying what to expect regarding IOV enable/disable on these cards.
i found no documentation or tool to enable this or i would not have done this
via ethtool.
it is not fun to waste weeks on troubleshooting sr-iov only to find out that it
was disabled in the cards eeprom.
it also doesn't help that when you lookup specs for this card it clearly lists
that it supports sr-iov, even the place i bought it from and their support said
it supported sr-iov (incorrectly, kind of)
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
