On 9/7/2023 5:10 PM, Eric Auger wrote: > Hi, > > On 9/7/23 09:16, Philippe Mathieu-Daudé wrote: >> Widening Cc to ARM/VFIO. >> >> On 4/8/23 11:15, Wu, Fei wrote: >>> On 8/3/2023 11:07 PM, Andrew Jones wrote: >>>> On Mon, Jul 31, 2023 at 09:53:17AM +0800, Fei Wu wrote: >>>>> riscv virt platform's memory started at 0x80000000 and >>>>> straddled the 4GiB boundary. Curiously enough, this choice >>>>> of a memory layout will prevent from launching a VM with >>>>> a bit more than 2000MiB and PCIe pass-thru on an x86 host, due >>>>> to identity mapping requirements for the MSI doorbell on x86, >>>>> and these (APIC/IOAPIC) live right below 4GiB. >>>>> >>>>> So just split the RAM range into two portions: >>>>> - 1 GiB range from 0x80000000 to 0xc0000000. >>>>> - The remainder at 0x100000000 >>>>> >>>>> ...leaving a hole between the ranges. >>>> >>>> Can you elaborate on the use case? Maybe provide details of the host >>>> system and the QEMU command line? I'm wondering why we didn't have >>>> any problems with the arm virt machine type. Has nobody tried this >>>> use case with that? Is the use case something valid for riscv, but >>>> not arm? >>>> >>> Firstly we have to enable pcie passthru on host, find the device groups, >>> e.g. the vga card, and add their pci ids to host kernel cmdline: >>> vfio-pci.ids=10de:0f02,10de:0e08 >>> >>> then start vm through qemu as follows: >>> $Q -machine virt -m 4G -smp 4 -nographic \ >>> -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \ >>> -kernel ./vmlinuz -initrd initrd.img -append "root=/dev/vda1 rw" \ >>> -drive >>> file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw >>> >>> \ >>> -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \ >>> -netdev user,id=vnet,hostfwd=:127.0.0.1:2223-:22 -device >>> virtio-net-pci,netdev=vnet >>> >>> Without this patch, qemu exits immediately instead of boots up. >>> >>> Just tried pcie passthru on arm, it cannot handle 4G memory either. >>> $Q -m 4G -smp 4 -cpu max -M virt -nographic \ >>> -pflash /usr/share/AAVMF/AAVMF_CODE.fd -pflash flash1.img \ >>> -drive if=none,file=ubuntu-22.04-server-cloudimg-arm64.img,id=hd0 \ >>> -device virtio-blk-device,drive=hd0 \ >>> -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 >>> >>> qemu-system-aarch64: -device vfio-pci,host=01:00.0: VFIO_MAP_DMA failed: >>> Invalid argument >>> qemu-system-aarch64: -device vfio-pci,host=01:00.0: vfio 0000:01:00.0: >>> failed to setup container for group 11: memory listener initialization >>> failed: Region mach-virt.ram: vfio_dma_map(0x55de3c2a97f0, 0x40000000, >>> 0x100000000, 0x7f8fcbe00000) = -22 (Invalid argument) > > The collision between the x86 host MSI reserved region [0xfee00000, > 0xfeefffff] and the ARM guest RAM starting at 1GB has also always > existed. But now this collision is properly detected instead of being > silenced. People have not really complained about this so far. Since the > existing guest RAM layout couldn't be changed, I am afraid we couldn't > do much. > Just as what this patch does for riscv, arm guest on x86 could do the same thing to adjust the guest RAM layout if necessary? This looks like a nice-to-have feature for arm if there still is a requirement to run arm guest on x86.
Thanks, Fei. > Eric >>> >>> Thanks, >>> Fei. >>> >>>> Thanks, >>>> drew >>> >>> >> >
