Relocation overflow on RISC-V with multi-range memory layout
Hi All,
I'm enabling PCIe passthrough on qemu riscv, the physical memory
range between 3GB and 4GB is reserved. Therefore if guest has 4GB ram,
two ranges are created as [2G, 3G) and [4G, 7G). More details can be
found here:
https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
When run grub.efi from uefi shell, a relocation problem happened in
grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
case R_RISCV_CALL:
case R_RISCV_CALL_PLT:
{
grub_uint32_t *abs_place = place;
grub_ssize_t off = sym_addr - (grub_addr_t) place;
grub_uint32_t hi20, lo12;
if (off != (grub_int32_t) off)
return grub_error (GRUB_ERR_BAD_MODULE, "relocation
overflow");
It requires `off' in the range of int32, but it's not enforced since the
>4GB memory can be used. I'm not familiar with grub, but this patch does
work for me:
--- a/include/grub/riscv64/efi/memory.h
+++ b/include/grub/riscv64/efi/memory.h
@@ -1,6 +1,6 @@
#ifndef GRUB_MEMORY_CPU_HEADER
#include
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
+#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
Any comments?
Thanks,
Fei.
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Relocation overflow on RISC-V with multi-range memory layout
On 9/25/2023 4:51 PM, Wu, Fei wrote:
> Hi All,
>
> I'm enabling PCIe passthrough on qemu riscv, the physical memory
> range between 3GB and 4GB is reserved. Therefore if guest has 4GB ram,
> two ranges are created as [2G, 3G) and [4G, 7G). More details can be
> found here:
> https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
>
> When run grub.efi from uefi shell, a relocation problem happened in
> grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
>
> case R_RISCV_CALL:
> case R_RISCV_CALL_PLT:
> {
> grub_uint32_t *abs_place = place;
> grub_ssize_t off = sym_addr - (grub_addr_t) place;
> grub_uint32_t hi20, lo12;
>
> if (off != (grub_int32_t) off)
> return grub_error (GRUB_ERR_BAD_MODULE, "relocation
> overflow");
>
> It requires `off' in the range of int32, but it's not enforced since the
>> 4GB memory can be used. I'm not familiar with grub, but this patch does
> work for me:
>
> --- a/include/grub/riscv64/efi/memory.h
> +++ b/include/grub/riscv64/efi/memory.h
> @@ -1,6 +1,6 @@
> #ifndef GRUB_MEMORY_CPU_HEADER
> #include
>
> -#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
> +#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
>
Anyone can help take a look? I will send it out for review if this is
the right fix.
The test I have done against commit db1faedcc:
qemu-system-riscv64 -nographic \
-M virt,pflash0=pflash0,pflash1=pflash1,acpi=off \
-m 3G -smp 2 \
-blockdev
node-name=pflash0,driver=file,read-only=on,filename=RISCV_VIRT_CODE.fd \
-blockdev node-name=pflash1,driver=file,filename=RISCV_VIRT_VARS.fd \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-device virtio-blk-device,drive=hd0 \
-drive file=fat:rw:/home/wufei/src/fat,id=hd0
then run grubriscv64.efi on uefi shell:
FS0:\> grubriscv64.efi
...
ProtectUefiImageCommon - 0xBED9E540
- 0xBDCCB000 - 0x004DF000
SetUefiImageMemoryAttributes - 0xBDCCB000 - 0x1000
(0x4000)
CpuSetMemoryAttributes: Set memory attributes not supported yet
SetUefiImageMemoryAttributes - 0xBDCCC000 - 0xC000
(0x0002)
CpuSetMemoryAttributes: Set memory attributes not supported yet
SetUefiImageMemoryAttributes - 0xBDCD8000 - 0x004D2000
(0x4000)
CpuSetMemoryAttributes: Set memory attributes not supported yet
InstallProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA 83FFF720
CpuSetMemoryAttributes: Set memory attributes not supported yet
relocation overflow
Aborted. Press any key to exit
Thanks,
Fei.
> Any comments?
>
> Thanks,
> Fei.
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Relocation overflow on RISC-V with multi-range memory layout
On 9/27/2023 11:23 PM, Vladimir 'phcoder' Serbinenko wrote:
> That is not the correct solution. Correct solution is to use trampoline
> facility like e.g. ppc does. Can you post the full reproduction
> instructions?
>
1. prepare the host for pcie passthrough, e.g. on ubuntu, put something
like the following to kernel cmd, the pci ids are the pci devices to
passthrough:
intel_iommu=on iommu=pt vfio-pci.ids=10de:0f02,10de:0e08
2. apply the patch mentioned below to qemu commit ccb86f079a9e4
3. run qemu as follows:
qemu-system-riscv64
-nographic \
-M virt,pflash0=pflash0,pflash1=pflash1,acpi=off \
-m 3G -smp 2 \
-blockdev
node-name=pflash0,driver=file,read-only=on,filename=RISCV_VIRT_CODE.fd \
-blockdev node-name=pflash1,driver=file,filename=RISCV_VIRT_VARS.fd \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-device virtio-blk-device,drive=hd0 \
-drive file=fat:rw:/home/wufei/src/fat,id=hd0
4. build and put grub.efi to the directory 'fat'
5. on uefi shell, run grub.efi
Thanks,
Fei.
> Le lun. 25 sept. 2023, 10:53, Wu, Fei <mailto:fei2...@intel.com>> a écrit :
>
> Hi All,
>
> I'm enabling PCIe passthrough on qemu riscv, the physical memory
> range between 3GB and 4GB is reserved. Therefore if guest has 4GB ram,
> two ranges are created as [2G, 3G) and [4G, 7G). More details can be
> found here:
>
> https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
>
> <https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/>
>
> When run grub.efi from uefi shell, a relocation problem happened in
> grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
>
> case R_RISCV_CALL:
> case R_RISCV_CALL_PLT:
> {
> grub_uint32_t *abs_place = place;
> grub_ssize_t off = sym_addr - (grub_addr_t) place;
> grub_uint32_t hi20, lo12;
>
> if (off != (grub_int32_t) off)
> return grub_error (GRUB_ERR_BAD_MODULE, "relocation
> overflow");
>
> It requires `off' in the range of int32, but it's not enforced since the
> >4GB memory can be used. I'm not familiar with grub, but this patch does
> work for me:
>
> --- a/include/grub/riscv64/efi/memory.h
> +++ b/include/grub/riscv64/efi/memory.h
> @@ -1,6 +1,6 @@
> #ifndef GRUB_MEMORY_CPU_HEADER
> #include
>
> -#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
> +#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
>
> Any comments?
>
> Thanks,
> Fei.
>
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org <mailto:Grub-devel@gnu.org>
> https://lists.gnu.org/mailman/listinfo/grub-devel
> <https://lists.gnu.org/mailman/listinfo/grub-devel>
>
>
> ___
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Relocation overflow on RISC-V with multi-range memory layout
On 10/9/2023 11:14 AM, Wu, Fei wrote:
> On 9/27/2023 11:23 PM, Vladimir 'phcoder' Serbinenko wrote:
>> That is not the correct solution. Correct solution is to use trampoline
>> facility like e.g. ppc does. Can you post the full reproduction
>> instructions?
>>
Hi Vladimir,
Could you please explain a little more about why this is not good, and
how does the trampoline address that? IIUC, x86_64 sets
GRUB_EFI_MAX_USABLE_ADDRESS to 0x for the same reason.
Thanks,
Fei.
> 1. prepare the host for pcie passthrough, e.g. on ubuntu, put something
> like the following to kernel cmd, the pci ids are the pci devices to
> passthrough:
> intel_iommu=on iommu=pt vfio-pci.ids=10de:0f02,10de:0e08
>
> 2. apply the patch mentioned below to qemu commit ccb86f079a9e4
> 3. run qemu as follows:
>
> qemu-system-riscv64
> -nographic \
> -M virt,pflash0=pflash0,pflash1=pflash1,acpi=off \
> -m 3G -smp 2 \
> -blockdev
> node-name=pflash0,driver=file,read-only=on,filename=RISCV_VIRT_CODE.fd \
> -blockdev node-name=pflash1,driver=file,filename=RISCV_VIRT_VARS.fd \
> -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
> -device virtio-blk-device,drive=hd0 \
> -drive file=fat:rw:/home/wufei/src/fat,id=hd0
>
> 4. build and put grub.efi to the directory 'fat'
> 5. on uefi shell, run grub.efi
>
> Thanks,
> Fei.
>
>> Le lun. 25 sept. 2023, 10:53, Wu, Fei > <mailto:fei2...@intel.com>> a écrit :
>>
>> Hi All,
>>
>> I'm enabling PCIe passthrough on qemu riscv, the physical memory
>> range between 3GB and 4GB is reserved. Therefore if guest has 4GB ram,
>> two ranges are created as [2G, 3G) and [4G, 7G). More details can be
>> found here:
>>
>> https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
>>
>> <https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/>
>>
>> When run grub.efi from uefi shell, a relocation problem happened in
>> grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
>>
>> case R_RISCV_CALL:
>> case R_RISCV_CALL_PLT:
>> {
>> grub_uint32_t *abs_place = place;
>> grub_ssize_t off = sym_addr - (grub_addr_t) place;
>> grub_uint32_t hi20, lo12;
>>
>> if (off != (grub_int32_t) off)
>> return grub_error (GRUB_ERR_BAD_MODULE, "relocation
>> overflow");
>>
>> It requires `off' in the range of int32, but it's not enforced since the
>> >4GB memory can be used. I'm not familiar with grub, but this patch does
>> work for me:
>>
>> --- a/include/grub/riscv64/efi/memory.h
>> +++ b/include/grub/riscv64/efi/memory.h
>> @@ -1,6 +1,6 @@
>> #ifndef GRUB_MEMORY_CPU_HEADER
>> #include
>>
>> -#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
>> +#define GRUB_EFI_MAX_USABLE_ADDRESS 0xULL
>>
>> Any comments?
>>
>> Thanks,
>> Fei.
>>
>> ___
>> Grub-devel mailing list
>> Grub-devel@gnu.org <mailto:Grub-devel@gnu.org>
>> https://lists.gnu.org/mailman/listinfo/grub-devel
>> <https://lists.gnu.org/mailman/listinfo/grub-devel>
>>
>>
>> ___
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> https://lists.gnu.org/mailman/listinfo/grub-devel
>
___
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Relocation overflow on RISC-V with multi-range memory layout
On 10/11/2023 9:50 PM, Vladimir 'phcoder' Serbinenko wrote:
>
>
> Le mer. 11 oct. 2023, 12:20, Wu, Fei <mailto:fei2...@intel.com>> a écrit :
>
> On 10/9/2023 11:14 AM, Wu, Fei wrote:
> > On 9/27/2023 11:23 PM, Vladimir 'phcoder' Serbinenko wrote:
> >> That is not the correct solution. Correct solution is to use
> trampoline
> >> facility like e.g. ppc does. Can you post the full reproduction
> >> instructions?
> >>
> Hi Vladimir,
>
> Could you please explain a little more about why this is not good, and
> how does the trampoline address that? IIUC, x86_64 sets
> GRUB_EFI_MAX_USABLE_ADDRESS to 0x for the same reason.
>
> x86_64 reason is different. It's because some EFI implementations don't
> map RAM above 4GiB contrary to the spec.
> Trampolines are small piece of code that is inserted by linker to
> provide absolute jump. Trampolines are always allocated together with
> the module and so they are reachable by relative jump. Basically what
> they do then is:
> ptr=&symbol;
> return (*ptr) (...);
> On most platforms that translates to a load+register jump. Let me see if
> I can do it quickly. Will you be able to test?
>
Thank you Vladimir. Sure, I can test it, let me know when your code is
ready.
Best Regards,
Fei.
>
> Thanks,
> Fei.
>
> > 1. prepare the host for pcie passthrough, e.g. on ubuntu, put
> something
> > like the following to kernel cmd, the pci ids are the pci devices to
> > passthrough:
> > intel_iommu=on iommu=pt vfio-pci.ids=10de:0f02,10de:0e08
> >
> > 2. apply the patch mentioned below to qemu commit ccb86f079a9e4
> > 3. run qemu as follows:
> >
> > qemu-system-riscv64
> > -nographic \
> > -M virt,pflash0=pflash0,pflash1=pflash1,acpi=off \
> > -m 3G -smp 2 \
> > -blockdev
> >
> node-name=pflash0,driver=file,read-only=on,filename=RISCV_VIRT_CODE.fd \
> > -blockdev node-name=pflash1,driver=file,filename=RISCV_VIRT_VARS.fd \
> > -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
> > -device virtio-blk-device,drive=hd0 \
> > -drive file=fat:rw:/home/wufei/src/fat,id=hd0
> >
> > 4. build and put grub.efi to the directory 'fat'
> > 5. on uefi shell, run grub.efi
> >
> > Thanks,
> > Fei.
> >
> >> Le lun. 25 sept. 2023, 10:53, Wu, Fei <mailto:fei2...@intel.com>
> >> <mailto:fei2...@intel.com <mailto:fei2...@intel.com>>> a écrit :
> >>
> >> Hi All,
> >>
> >> I'm enabling PCIe passthrough on qemu riscv, the physical memory
> >> range between 3GB and 4GB is reserved. Therefore if guest has
> 4GB ram,
> >> two ranges are created as [2G, 3G) and [4G, 7G). More details
> can be
> >> found here:
> >>
>
> https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
>
> <https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/>
>
> <https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/
>
> <https://lore.kernel.org/all/cakmqykmtazt5sacumd4vxyfgaqibpzqjahttsusb+yekhcy...@mail.gmail.com/T/>>
> >>
> >> When run grub.efi from uefi shell, a relocation problem
> happened in
> >> grub_arch_dl_relocate_symbols() of grub-core/kern/riscv/dl.c:
> >>
> >> case R_RISCV_CALL:
> >> case R_RISCV_CALL_PLT:
> >> {
> >> grub_uint32_t *abs_place = place;
> >> grub_ssize_t off = sym_addr - (grub_addr_t) place;
> >> grub_uint32_t hi20, lo12;
> >>
> >> if (off != (grub_int32_t) off)
> >> return grub_error (GRUB_ERR_BAD_MODULE, "relocation
> >> overflow");
> >>
> >> It requires `off' in the range of int32, but it's not
> enforced since the
> >> >4GB memory can be used. I'm not familiar with grub, but this
> patch does
> >> work for me:
> >>
> >> --- a/include/grub/riscv64/efi/memory.h
> >> +++ b/include/grub/riscv64/efi/memory.h
> >> @@ -1,6 +1,6 @@
>
Re: Relocation overflow on RISC-V with multi-range memory layout
On 10/15/2023 5:16 AM, Vladimir 'phcoder' Serbinenko wrote: > I looked into it and found out that current code misses both got and > trampolines. I have a template for solution but I didn't test it yet. > Can you upload your EFI images (RISCV_VIRT_CODE.fd and > RISCV_VIRT_VARS.fd) somewhere so I don't have to compile myself? > https://github.com/phcoder/GRUB/tree/riscv > Do you have any update on this? Please let me know if you want me to have a try. Currently this branch doesn't work for me. Loading driver at 0x000BDCCD000 EntryPoint=0x000BDCCE000 ed yet InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF BED98A18(0x4000) ProtectUefiImageCommon - 0xBED60040attributes not supported yet - 0xBDCCD000 - 0x004DD000C-A22A-E5F46812F4CA 83FFF720 SetUefiImageMemoryAttributes - 0xBDCCD000 - 0x1000 (0x4000) CpuSetMemoryAttributes: Set memory attributes not supported yet SetUefiImageMemoryAttributes - 0xBDCCE000 - 0xC000 (0x0002) CpuSetMemoryAttributes: Set memory attributes not supported yet SetUefiImageMemoryAttributes - 0xBDCDA000 - 0x004D (0x4000) CpuSetMemoryAttributes: Set memory attributes not supported yet InstallProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA 83FFF720 RISCV64 Exception Type - 0005(EXCEPT_RISCV_LOAD_ACCESS_FAULT) t0 = 0x00047t1 = 0x083FFF370 t2 = 0x00043t3 = 0x0752F3136 t4 = 0x04E16t5 = 0x04FDC t6 = 0x0002510030021s0 = 0x083FFF5F0 s1 = 0x00013s2 = 0x0 s3 = 0x02000s4 = 0x0 s5 = 0x0BDCDB0C0s6 = 0x0800A6800 s7 = 0x0007Fs8 = 0x08001A038 s9 = 0x08002AAB0 s10 = 0x0 s11 = 0x0a0 = 0x0BDCDB0C0 a1 = 0x0BFFFE018a2 = 0x0BDCD4F12 a3 = 0x0BED60E98a4 = 0x0BFFFE018 a5 = 0x0AFAFAFAFAFAFAFAFa6 = 0x04FDC a7 = 0x000A2 zero = 0x0 ra = 0x0BDCD2FA4sp = 0x00010 gp = 0x0tp = 0x080036000 sepc = 0x0BDCD1D9A sstatus = 0x080026120 stval = 0x0AFAFAFAFAFAFAFAF Thanks, Fei. > On Thu, Oct 12, 2023 at 2:30 AM Wu, Fei wrote: >> >> On 10/11/2023 9:50 PM, Vladimir 'phcoder' Serbinenko wrote: >>> >>> >>> Le mer. 11 oct. 2023, 12:20, Wu, Fei >> <mailto:fei2...@intel.com>> a écrit : >>> >>> On 10/9/2023 11:14 AM, Wu, Fei wrote: >>> > On 9/27/2023 11:23 PM, Vladimir 'phcoder' Serbinenko wrote: >>> >> That is not the correct solution. Correct solution is to use >>> trampoline >>> >> facility like e.g. ppc does. Can you post the full reproduction >>> >> instructions? >>> >> >>> Hi Vladimir, >>> >>> Could you please explain a little more about why this is not good, and >>> how does the trampoline address that? IIUC, x86_64 sets >>> GRUB_EFI_MAX_USABLE_ADDRESS to 0x for the same reason. >>> >>> x86_64 reason is different. It's because some EFI implementations don't >>> map RAM above 4GiB contrary to the spec. >>> Trampolines are small piece of code that is inserted by linker to >>> provide absolute jump. Trampolines are always allocated together with >>> the module and so they are reachable by relative jump. Basically what >>> they do then is: >>> ptr=&symbol; >>> return (*ptr) (...); >>> On most platforms that translates to a load+register jump. Let me see if >>> I can do it quickly. Will you be able to test? >>> >> Thank you Vladimir. Sure, I can test it, let me know when your code is >> ready. >> >> Best Regards, >> Fei. >> >>> >>> Thanks, >>> Fei. >>> >>> > 1. prepare the host for pcie passthrough, e.g. on ubuntu, put >>> something >>> > like the following to kernel cmd, the pci ids are the pci devices to >>> > passthrough: >>> > intel_iommu=on iommu=pt vfio-pci.ids=10de:0f02,10de:0e08 >>> > >>> > 2. apply the patch mentioned below to qemu commit ccb86f079a9e4 >>> > 3. run qemu as follows: >>> > >>> > qemu-system-riscv64 >>> &
