CC-in a few people who can be interested in this... On Wed, Jun 30, 2021 at 06:40:11PM +1000, Daniel Axtens wrote: > On powerpc-ieee1275, we are running out of memory trying to verify > anything. This is because: > > - we have to load an entire file into memory to verify it. This is > extremely difficult to change with appended signatures. > - We only have 32MB of heap. > - Distro kernels are now often around 30MB. > > So we want to claim more memory from OpenFirmware for our heap.
AFAICT it is common problem in the GRUB right now. Please take a look at [1], [2]. It would be nice to find general solution for all. Of course if possible. So, if somebody could take a look at memory management in the GRUB and propose a solution for the problem that would be perfect. Any volunteers? Daniel [1] https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00009.html [2] https://lists.gnu.org/archive/html/grub-devel/2021-01/msg00031.html > There are some complications: > > - The grub mm code isn't the only thing that will make claims on > memory from OpenFirmware: > > * PFW/SLOF will have claimed some for their own use. > > * The ieee1275 loader will try to find other bits of memory that we > haven't claimed to place the kernel and initrd when we go to boot. > > * Once we load Linux, it will also try to claim memory. It claims > memory without any reference to /memory/available, it just starts > at min(top of RMO, 768MB) and works down. So we need to avoid this > area. See arch/powerpc/kernel/prom_init.c as of v5.11. > > - The smallest amount of memory a ppc64 KVM guest can have is 256MB. > It doesn't work with distro kernels but can work with custom kernels. > We should maintain support for that. (ppc32 can boot with even less, > and we shouldn't break that either.) > > - Even if a VM has more memory, the memory OpenFirmware makes available > as Real Memory Area can be restricted. A freshly created LPAR on a > PowerVM machine is likely to have only 256MB available to OpenFirmware > even if it has many gigabytes of memory allocated. > > EFI systems will attempt to allocate 1/4th of the available memory, > clamped to between 1M and 1600M. That seems like a good sort of > approach, we just need to figure out if 1/4 is the right fraction > for us. > > We don't know in advance how big the kernel and initrd are going to be, > which makes figuring out how much memory we can take a bit tricky. > > To figure out how much memory we should leave unused, I looked at: > > - an Ubuntu 20.04.1 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~50MB > > - a RHEL8.2 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~30MB > > Ubuntu VMs struggle to boot with just 256MB under SLOF. > RHEL likewise has a higher minimum supported memory figure. > So lets first consider a distro kernel and 512MB of addressible memory. > (This is the default case for anything booting under PFW.) Say we lose > 131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB > is ~95MB. That should be enough to verify a 30MB vmlinux and should > leave plenty of space to load Linux and the initrd. > > If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4 > of that is a smidge under 32MB, which gives us very poor odds of verifying > a distro-sized kernel. However, if we need 80MB just to put the kernel > and initrd in memory, we can't claim any more than 45MB anyway. So 1/4 > will do. We'll come back to this later. > > grub is always built as a 32-bit binary, even if it's loading a ppc64 > kernel. So we can't address memory beyond 4GB. This gives a natural cap > of 1GB for powerpc-ieee1275. > > Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap. > > make check still works for both i386 and powerpc and I've booted > powerpc grub with this change under SLOF and PFW. > > Signed-off-by: Daniel Axtens <d...@axtens.net> > --- > docs/grub-dev.texi | 6 ++- > grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++------- > 2 files changed, 69 insertions(+), 18 deletions(-) > > diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi > index 6c629a23e2dc..c11f1ac46de2 100644 > --- a/docs/grub-dev.texi > +++ b/docs/grub-dev.texi > @@ -1047,7 +1047,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI > for its heap, at most > 1.6 GiB. > > On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275. > -It allocates at most 32MiB for its heap. > + > +On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On > +powerpc-ieee1275, GRUB allocates up to 1GiB. > > On sparc64-ieee1275 stack is 256KiB and heap is 2MiB. > > @@ -1075,7 +1077,7 @@ In short: > @item i386-qemu @tab 60 KiB @tab < 4 GiB > @item *-efi @tab ? @tab < 1.6 GiB > @item i386-ieee1275 @tab ? @tab < 32 MiB > -@item powerpc-ieee1275 @tab ? @tab < 32 MiB > +@item powerpc-ieee1275 @tab ? @tab < 1 GiB > @item sparc64-ieee1275 @tab 256KiB @tab 2 MiB > @item arm-uboot @tab 256KiB @tab 2 MiB > @item mips(el)-qemu_mips @tab 2MiB @tab 253 MiB > diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c > index c5d091689f29..4162b5949df4 100644 > --- a/grub-core/kern/ieee1275/init.c > +++ b/grub-core/kern/ieee1275/init.c > @@ -45,11 +45,12 @@ > #include <grub/machine/kernel.h> > #endif > > -/* The maximum heap size we're going to claim */ > +/* The maximum heap size we're going to claim. Not used by sparc. > + We allocate 1/4 of the available memory under 4G, up to this limit. */ > #ifdef __i386__ > #define HEAP_MAX_SIZE (unsigned long) (64 * 1024 * 1024) > -#else > -#define HEAP_MAX_SIZE (unsigned long) (32 * 1024 * 1024) > +#else // __powerpc__ > +#define HEAP_MAX_SIZE (unsigned long) (1 * 1024 * 1024 * 1024) > #endif > > extern char _start[]; > @@ -145,16 +146,45 @@ grub_claim_heap (void) > + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000); > } > #else > -/* Helper for grub_claim_heap. */ > +/* Helper for grub_claim_heap on powerpc. */ > +static int > +heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > + void *data) > +{ > + grub_uint32_t total = *(grub_uint32_t *)data; > + > + if (type != GRUB_MEMORY_AVAILABLE) > + return 0; > + > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > + total += len; > + *(grub_uint32_t *)data = total; > + > + return 0; > +} > + > static int > heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > void *data) > { > - unsigned long *total = data; > + grub_uint32_t total = *(grub_uint32_t *)data; > > if (type != GRUB_MEMORY_AVAILABLE) > return 0; > > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM)) > { > if (addr + len <= 0x180000) > @@ -168,10 +198,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, > grub_memory_type_t type, > } > len -= 1; /* Required for some firmware. */ > > - /* Never exceed HEAP_MAX_SIZE */ > - if (*total + len > HEAP_MAX_SIZE) > - len = HEAP_MAX_SIZE - *total; > - > /* In theory, firmware should already prevent this from happening by not > listing our own image in /memory/available. The check below is intended > as a safeguard in case that doesn't happen. However, it doesn't protect > @@ -183,6 +209,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, > grub_memory_type_t type, > len = 0; > } > > + /* If this block contains 0x30000000 (768MB), do not claim below that. > + Linux likes to claim memory at min(RMO top, 768MB) and works down > + without reference to /memory/available. */ > + if ((addr < 0x30000000) && ((addr + len) > 0x30000000)) > + { > + len = len - (0x30000000 - addr); > + addr = 0x30000000; > + } > + > + if (len > total) > + len = total; > + > if (len) > { > grub_err_t err; > @@ -191,10 +229,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, > grub_memory_type_t type, > if (err) > return err; > grub_mm_init_region ((void *) (grub_addr_t) addr, len); > + total -= len; > } > > - *total += len; > - if (*total >= HEAP_MAX_SIZE) > + *(grub_uint32_t *)data = total; > + > + if (total == 0) > return 1; > > return 0; > @@ -203,13 +243,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, > grub_memory_type_t type, > static void > grub_claim_heap (void) > { > - unsigned long total = 0; > + grub_uint32_t total = 0; > > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM)) > - heap_init (GRUB_IEEE1275_STATIC_HEAP_START, > GRUB_IEEE1275_STATIC_HEAP_LEN, > - 1, &total); > - else > - grub_machine_mmap_iterate (heap_init, &total); > + { > + heap_init (GRUB_IEEE1275_STATIC_HEAP_START, > GRUB_IEEE1275_STATIC_HEAP_LEN, > + 1, &total); > + return; > + } > + > + grub_machine_mmap_iterate (heap_size, &total); > + > + total = total / 4; > + if (total > HEAP_MAX_SIZE) > + total = HEAP_MAX_SIZE; > + > + grub_machine_mmap_iterate (heap_init, &total); > } > #endif > > -- > 2.30.2 _______________________________________________ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
