On 20.05.2023 00:44, Stefano Stabellini wrote: > On Fri, 19 May 2023, Jan Beulich wrote: >> On 18.05.2023 12:34, Roger Pau Monné wrote: >>> On Wed, May 17, 2023 at 05:59:31PM -0700, Stefano Stabellini wrote: >>>> I have run into another PVH Dom0 issue. I am trying to enable a PVH Dom0 >>>> test with the brand new gitlab-ci runner offered by Qubes. It is an AMD >>>> Zen3 system and we already have a few successful tests with it, see >>>> automation/gitlab-ci/test.yaml. >>>> >>>> We managed to narrow down the issue to a console problem. We are >>>> currently using console=com1 com1=115200,8n1,pci,msi as Xen command line >>>> options, it works with PV Dom0 and it is using a PCI UART card. >>>> >>>> In the case of Dom0 PVH: >>>> - it works without console=com1 >>>> - it works with console=com1 and with the patch appended below >>>> - it doesn't work otherwise and crashes with this error: >>>> https://matrix-client.matrix.org/_matrix/media/r0/download/invisiblethingslab.com/uzcmldIqHptFZuxqsJtviLZK >>> >>> Jan also noticed this, and we have a ticket for it in gitlab: >>> >>> https://gitlab.com/xen-project/xen/-/issues/85 >>> >>>> What is the right way to fix it? >>> >>> I think the right fix is to simply avoid hidden devices from being >>> handled by vPCI, in any case such devices won't work propewrly with >>> vPCI because they are in use by Xen, and so any cached information by >>> vPCI is likely to become stable as Xen can modify the device without >>> vPCI noticing. >>> >>> I think the chunk below should help. It's not clear to me however how >>> hidden devices should be handled, is the intention to completely hide >>> such devices from dom0? >> >> No, Dom0 should still be able to see them in a (mostly) r/o fashion. > > But why? If something is in-use by Xen (e.g. IOMMU, a serial PCI device, > etc.) ideally Dom0 shouldn't even know of its existence because the > device is not exposed to Dom0. Dom0 is not meant to use it. Why let Dom0 > know it exists if Dom0 should not use it?
Because we want to disturb the topology Dom0 sees as little as possible. For example, imagine the device is func 0 of a multi-function device. How would Dom0 know to even look at the other functions, when it can't even read the relevant bit in func 0's config space? Jan > In Xen on ARM, initially we didn't expose devices used by Xen to Dom0 > at all. However to hide them completely we had to make complex device > tree manipulations. Now instead we leave the device nodes in device tree > as is, but we change the "status" property to "disabled". > > The idea is still that we completely hide Xen devices from Dom0, but > because of implementation complexity, instead of completing taking away > the corresponding nodes from device tree, we change them to disabled, > which still leads to the same result: the guest OS will skip them. > > I am saying this without being familiar with the x86 PVH implementation, > so pardon my ignorance here, but it seems to me that as we are moving > toward an better architecture with PVH, once that allows any OS to be > Dom0, not just Linux, we would want to also completely hide devices > owned by Xen from Dom0. > > That way we don't need any workaround in the guest OS for it not to use > them.
