> From: Michael Kelley <[email protected]> > Sent: Sunday, April 5, 2026 4:15 PM > > ... > > Note: we still need to figure out how to address the possible MMIO > > conflict between hyperv_drm and pci_hyperv in the case of 32-bit PCI > > MMIO BARs, but that's of low priority because all PCI devices available > > to a Linux VM on Azure or on a modern host should use 64-bit BARs and > > should not use 32-bit BARs -- I checked Mellanox VFs, MANA VFs, NVMe > > devices, and GPUs in Linux VMs on Azure, and found no 32-bit BARs. > > Just to clarify, since this patch is predicated on all BARs being 64-bit, > hv_pci_alloc_bridge_windows() never encounters a non-zero > hbus->low_mmio_space, and hence also never allocates from low > MMIO space. So hv_pci_alloc_bridge_windows() does not need to be > patched. Is that correct?
Correct. For 32-bit BARs (if any), IMO we can't really do anything for them in hv_pci_allocate_bridge_windows(), since they must reside below 4GB. Note: while the patch doesn't fix the MMIO conflict if there are any 32-bit BARs, the patch doesn't make things worse for 32-bit BARs (if any). > Taking a broader view, fundamentally the current MMIO location of > the frame buffer may be unknown to the Linux guest. At the same time, > Linux must ensure that PCI devices don't get assigned to the MMIO space > where the frame buffer is located. While the current MMIO location of > the frame buffer may be unknown, we can assume it was placed in low > MMIO space by the host -- either Windows Hyper-V or Linux/VMM > in the root partition, and perhaps as mediated by a paravisor. Probably > need to confirm with the Linux-in-the-root partition team (and maybe > the OpenHCL team) that this assumption is true. IMO this is a good idea! It looks like the framebuffer base always starts at the beginning of the low MMIO space. We can reserve some MMIO for the framebuffer at the beginning of the low MMIO space. > Presumably the > hyperv_drm driver doesn't need to move the frame buffer, but if it > does, it must stay in the low MMIO space. It looks like this assumption is true. > This patch depends on this assumption, and effectively reserves > the entire low MMIO space for the frame buffer. To make it precise, the patch reserves the entire low MMIO space for the frame buffer and the 32-bit BARs (if any), and there is no MMIO conflict in the first kernel (assuming hyperv_drm doesn't relocate the MMIO range), and there can be an MMIO conflict in the kdump/kexec kernel if there is any 32-bit BAR. > The low MMIO space > size defaults to 128 MiB on a local Hyper-V, Yes, by default, the low MMIO base =0xf800_0000, size=128MB, but the range [0xfed4_0000, 0xffff_ffff], whose size is 18.75MB, is reserved for vTPM: see vmbus_walk_resources(). So by default the available low MMIO size for hyperv_drm is 128 - 18.75 = 109.25 MB. The size of the framebuffer should be aligned to 2MB, so if the framebuffer size is bigger than 108MB, it looks like there is no enough MMIO space in the low MMIO range, e.g. with the below command: Set-VMVideo -VMName vm_name -HorizontalResolution 7680 -VerticalResolution 4320 -ResolutionType Maximum , the resulting max framebuffer size is 7680 * 4320 * 32/8 /1024.0/1024 = 126.5625, which would be rounded up to 128MB. However, according to my testing, with the above command, the low MMIO base = 0xf000_0000, size=256MB, so it's probably ok to reserve 128 MB for the frame buffer. In case the low MMIO size is <=64MB, we would want to reserve less MMIO for the frame buffer. > and is set to 3 GiB in most > Azure VMs (or to 1 GiB in an Azure CVM), so that all gets reserved. > > A slightly different approach to the whole problem is to change > vmbus_reserve_fb(). If it is unable to get a non-zero "start" value, then > it should use the same assumption as above, and reserve a frame buffer > area starting at the lowest address in low MMIO space. The reserved size > could be the max possible frame buffer size, which I think is 64 MiB (?). It can be 128MB with the highest resolution 7680*4320 (I hope the highest resolution won't become bigger in the future). > This still leaves low MMIO space for subsequent PCI devices, and allows > 32-bit BARs to continue to work. This approach requires one further > assumption, which is that the host, plus any movement by hyperv_drm, > has kept the frame buffer at the low end of the low MMIO space. From > what I've seen, that assumption is reality -- the frame buffer always > starts at the beginning of low MMIO space. > > This approach could be taken one step further, where vmbus_reserve_fb() > *always* reserves 64 MiB starting at the low end of low MMIO space, > regardless of the value of "start". The messy code for getting "start" > could be dropped entirely, and the dependency on CONFIG_SYSFB goes > away. Or maybe still get the value of "start" and "size", and if non-zero > just do a sanity check that they are within the fixed 64 MiB reserved area. > > Thoughts? To me tweaking vmbus_reserve_fb() is a more > straightforward and explicit way to do the reserving, vs. modifying > the requested range in the Hyper-V PCI driver. Agreed. Let me try to make a new patch for review. > And FWIW, it avoids introducing the 32-bit BAR limitation. This patch addresses the MMIO conflict for 64-bit BARs and not for 32-bit BARs (if any). The patch does not introduce the 32-bit BAR limitation. Thanks, -- Dexuan
