Actually have a Xeon setup where ootb all iommu groups were sane and separated, I just use the second NIC on my motherboard for the Windows vm as virtio makes it a bottleneck
On Aug 13, 2017 8:19 AM, "Torbjorn Jansson" < torbjorn.jans...@mbox200.swipnet.se> wrote: > On 2017-08-08 00:40, Alex Williamson wrote: > >> On Mon, 24 Jul 2017 09:35:40 +0200 >> Torbjorn Jansson <torbjorn.jans...@mbox200.swipnet.se> wrote: >> >> Hello. >>> >>> i'm considering upgrading my linux box that i use for virtualization >>> (i5-3470, >>> Z77 mb). >>> at the moment i have a successfully working setup where i pass thru a >>> gtx970 to >>> a windows vm. >>> i also have an aging 4 port network card, see below: >>> ------ >>> 04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet >>> Controller (Copper) (rev 06) >>> 04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet >>> Controller (Copper) (rev 06) >>> 05:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet >>> Controller (Copper) (rev 06) >>> 05:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet >>> Controller (Copper) (rev 06) >>> ------ >>> >> >> >> Interestingly, we have quirks for most of the 82571 cards, but not the >> quad port because the onboard switch doesn't support ACS so everything >> would get grouped back together anyway. >> >> > > yes i have noticed they all get grouped together no matter what i do. > > > i was thinking of replacing this card since i at the moment can't assign >>> this >>> in a good way, my only option now is to assign all 4 ports to same vm >>> which >>> doesn't really help. >>> >> >> A dual port version of the same might give your two NICs in separate >> groups depending on the ACS capabilities/quirks where it's installed >> if you're looking for a cheap option. >> >> > > problem is, i will most likely need more than 2 ports. > > > so, i was thinking of getting a x99 based mb, a xeon e5 v4 cpu and a >>> I350-T2 >>> network card that supports sr-iov. >>> >>> question is, how do i know that the main board also have the necessary >>> bits for >>> sr-iov support? >>> as i understand it, the bios have to do something too to make it work. >>> >>> >>> main boards i was considering was MSI X99A gaming 7 and also ASUS X99-E >>> WS. >>> the msi board have a nice block diagram that says how things is >>> connected and >>> manual looks abit better >>> >>> asus manual i don't really get, i didn't find any clear indication of >>> how many >>> lanes you get on the slots and what is listed is a bit confusing. >>> >>> my thinking was that maybe a workstation mb have higher probability of >>> better >>> support for more advanced features like this. >>> but this might be an incorrect assumption. >>> >>> any advice would be welcomed. >>> >> >> An SR-IOV card needs to be installed into an ACS capable/quirked slot >> or everything will be grouped together and it'd be pointless. The PCH >> slot on your Z77 (if it has one) would meet this requirement. Beyond >> that, ARI support can play a factor in the bus number layout and if it >> exists can make something like an 82756/i350 work without BIOS support >> given its scant SR-IOV resource requirements. Without it, extra bus >> numbers might need to be allocated which requires BIOS support, but >> Linux can do as well with a pci=assign-buses boot options (pci=realloc >> is also a useful options). For a 1Gbps NIC, you're not really losing >> anything by installing it into a PCH root port slot, so I'd give that a >> shot on your current board unless you're just using this as an excuse to >> upgrade the whole system. Thanks, >> >> > thanks for the advice. > i decided to go ahead with my upgrade, the reason was not only the sr-iov > part but mainly to get more cores and memory. > i just want to make sure i can use the sr-iov part too while i'm upgrading. > > i decided to go for the "MSI X99A gaming 7" mainboard, but unfortunately i > didn't have much luck in getting sr-iov working. > booting with pci=realloc,assign-busses doesn't help. > only difference i see in dmesg is things like: > [ 1.432766] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03 > [ 1.432915] pci_bus 0000:04: busn_res: can not insert [bus 04-ff] under > [bus 00-fe] (conflicts with (null) [bus 00-fe]) > > so i suspect something is wrong. > i have attached some logs. > > i also tried asking msi support about this and they were nice enough to > create a custom bios where sr-iov is supposed to be enabled but > unfortunately i still didn't have much luck. > > the mainboard in question is a desktop board and not a server board so i > can kind of understand that it isn't working since sr-iov is not something > your average desktop user is going to need or ask for. > > anyway, any input is appreciated. > > _______________________________________________ > vfio-users mailing list > vfio-users@redhat.com > https://www.redhat.com/mailman/listinfo/vfio-users > >
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
