On Thu, 20 Oct 2016 13:01:07 +0200 Ward Poelmans <ward.poelm...@ugent.be> wrote:
> On 19-10-16 16:33, Alex Williamson wrote: > > On Wed, 19 Oct 2016 16:03:59 +0200 > > Ward Poelmans <ward.poelm...@ugent.be> wrote: > >> > >> I've looked at the AMDGPU module and I think it checks a table in the > >> vbios to determine if it can do SRIOV. Upon testing it, the flag 'vBIOS > >> is sr-iov ready' is set but the flag 'sr-iov is enabled on this GPU' is > >> not. > >> > >> Anybody any idea how to activate this? It works with vmware, so I should > >> be possible to do it with qemu too. > > > > Assuming they've implemented standard SR-IOV support in Linux, you > > should find several sriov files in /sys/bus/pci/devices/$ADDRESS/, > > namely srio_numvfs and sriov_totalvfs. totalvfs should match the > > SR-IOV capability and report 16, numvfs will report 0 and you can echo > > values into numvfs to set the number of enabled VFs. Note that due to > > the way SR-IOV works, it is not possible change from one non-zero > > number of VFs to another, for example adding one if VFs are already > > enabled. numvfs must be set to zero before any new non-zero value can > > be set. Let us know what you find, I haven't handled these cards > > myself nor heard of users who have. Thanks, > > Hi Alex, > > I've already tried that and it doesn't work. The files are there but > upon writing you get: > pci 0000:04:00.0: Driver doesn't support SRIOV configuration via sysfs > > I've been looking in the amdgpu driver and the header 'amdgpu_virt.h' > [1] seems to point to some form of support. Currently, the virtual_caps > is set to 1 (which means the vbios support sr-iov but it's not enabled). > > So, I've tried to write to the pcie register to flip the sr-iov to enabled: > setpci -v -s 04:00.0 ECAP_SRIOV+8.B=19 > gives us: > IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy+ > > Using the 'srio_numvfs' file still fails but if we now load the amdgpu > driver, the second bit of virtual_caps is also on (meaning sr-iov is > enabled). However, the driver fails with 'Fatal error during GPU init' > (full output at [2]). It seems like the module might be doing too much? > I think it should not do a full Post init if you want virtual functions? > > We've also had a look at what AMD provides to activate it with vmware > but there seems hardly to be anything there. It has some pci id's and > some python scripts, but no goodies. > > Any other ideas? Nope, seems like AMD hasn't bothered to enable SR-IOV support for Linux yet, there are clearly no calls to pci_enable_sriov() from the amdgpu driver. There's typically PF driver integration work for VF support, enabling SR-IOV in the PF capability might make VFs appear on the bus, but they're not likely to work. Complain to AMD I guess. Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
