Hi,
On 17/07/2020 16:47, Bertrand Marquis wrote:
On 17 Jul 2020, at 17:26, Julien Grall <jul...@xen.org> wrote:
On 17/07/2020 15:47, Bertrand Marquis wrote:
pci=[ "PCI_SPEC_STRING", "PCI_SPEC_STRING", ...]
Guest will be only able to access the assigned devices and see the bridges.
Guest will not be able to access or see the devices that are no assigned to him.
Limitation:
* As of now all the bridges in the PCI bus are seen by the guest on the VPCI
bus.
Why do you want to expose all the bridges to a guest? Does this mean that the
BDF should always match between the host and the guest?
That’s not really something that we wanted but this was the easiest way to go.
As said in a previous mail we could build a VPCI bus with a completely
different topology but I am not sure of the advantages this would have.
Do you see some reason to do this ?
Yes :):
1) If a platform has two host controllers (IIRC Thunder-X has it) then you
would need to expose two host controllers to your guest. I think this is
undesirable if your guest is only using a couple of PCI devices on each host
controllers.
2) In the case of migration (live or not), you may want to use a difference
PCI card on the target platform. So your BDF and bridges may be different.
Therefore I think the virtual topology can be beneficial.
I would see a big advantage definitely to have only one VPCI bus per guest and
put all devices in their independently of the hardware domain the device is on.
But this will probably make the VPCI BARs value computation a bit more complex
as we might end up with no space on the guest physical map for it.
This might make the implementation a lot more complex.
I am not sure to understand your argument about the space... You should
be able to find out the size of each BARs, so you can size the MMIO
window correctly. This shouldn't add a lot of complexity.
I am not asking any implementation for this, but we need to make sure
the design can easily be extended for other use cases. In the case of
server, we will likely want to expose a single vPCI to the guest.
- Is there any memory access that can bypassed the IOMMU (e.g doorbell)?
This is still something to be investigated as part of the MSI implementation.
If you have any idea here, feel free to tell us.
My memory is a bit fuzzy here. I am sure that the doorbell can bypass the IOMMU
on some platform, but I also vaguely remember that accesses to the PCI host
controller memory window may also bypass the IOMMU. A good reading might be [2].
IIRC, I came to the conclusion that we may want to use the host memory map in
the guest when using the PCI passthrough. But maybe not on all the platforms.
Definitely a lot of this would be easier if could use 1:1 mapping.
We will keep that in mind when we will start to investigate on the MSI part.
Hmmm... Maybe I wasn't clear enough but the problem is not only
happening with MSIs doorbells. It is also with the P2P transactions.
Again, I am not asking to implement it at the beginning. However, it
would be good to outline the potential limitations of the approach in
your design.
Cheers,
--
Julien Grall
