On 08/19/2015 12:01 PM, Wei Yang wrote:
In original design, it tries to group VFs to enable more number of VFs in the
system, when VF BAR is bigger than 64MB. This design has a flaw in which one
error on a VF will interfere other VFs in the same group.
This patch series change this design by using M64 BAR in Single PE mode to
cover only one VF BAR. By doing so, it gives absolute isolation between VFs.
With or without this patchset, this fails with a horrible loop of EEHs:
rmmod mlx4_en mlx4_ib mlx4_core
modprobe mlx4_core num_vfs=4 probe_vf=4 port_type_array=2,2 debug_level=1
No guest is needed, just boot and do these commands. The EEH error is
pointing to a broken DMA address. iommu=nobypass fixed it for 4 VFs case
but when I try 16 VFs, none is created.
What is the correct base tree and what hardware did you use for the testing
_exactly_?
Mine is "Ethernet controller: Mellanox Technologies MT27520 Family
[ConnectX-3 Pro]" with 128MB BARs and that works (just double checked - can
create all 16 VFs) with PowerKVM 3.1 so it is not a hardware issue.
v4:
* rebase the code on top of v4.2-rc7
* switch back to use the dynamic version of pe_num_map and m64_map
* split the memory allocation and PE assignment of pe_num_map to make it
more easy to read
* check pe_num_map value before free PE.
* add the rename reason for pe_num_map and m64_map in change log
v3:
* return -ENOSPC when a VF has non-64bit prefetchable BAR
* rename offset to pe_num_map and define it staticly
* change commit log based on comments
* define m64_map staticly
v2:
* clean up iov bar alignment calculation
* change m64s to m64_bars
* add a field to represent M64 Single PE mode will be used
* change m64_wins to m64_map
* calculate the gate instead of hard coded
* dynamically allocate m64_map
* dynamically allocate PE#
* add a case to calculate iov bar alignment when M64 Single PE is used
* when M64 Single PE is used, compare num_vfs with M64 BAR available number
in system at first
Wei Yang (6):
powerpc/powernv: don't enable SRIOV when VF BAR has non
64bit-prefetchable BAR
powerpc/powernv: simplify the calculation of iov resource alignment
powerpc/powernv: use one M64 BAR in Single PE mode for one VF BAR
powerpc/powernv: replace the hard coded boundary with gate
powerpc/powernv: boundary the total VF BAR size instead of the
individual one
powerpc/powernv: allocate sparse PE# when using M64 BAR in Single PE
mode
arch/powerpc/include/asm/pci-bridge.h | 7 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 328 +++++++++++++++--------------
2 files changed, 175 insertions(+), 160 deletions(-)
--
Alexey
_______________________________________________
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
