Hi,
On 3/8/19 10:10 AM, Amit Tomer wrote:
Yes, you are right. I made a couple of quick fixes for this issue and
another issue raised by Julien during review (the usage of p2m_ram_t).
Amit, if you would like to give it a try I have a work-in-progress
branch with the fixes here:
We did try new branch with new fixes but we see some other crash now.
XEN) Loading kernel from boot module @ 000000007a000000
(XEN) Allocating 1:1 mappings totalling 2048MB for dom0:
(XEN) No bank has been allocated below 4GB.
(XEN) BANK[0] 0x00000500000000-0x00000540000000 (1024MB)
(XEN) BANK[1] 0x00000600000000-0x00000640000000 (1024MB)
(XEN) Grant table range: 0x0000073fe00000-0x0000073fe40000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading zImage from 000000007a000000 to 0000000500080000-0000000501880000
(XEN) Loading dom0 DTB to 0x0000000508000000-0x00000005080117d0
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing free RAM on in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 292kB init memory.
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Boot CPU: AArch64 Processor [411fd073]
[ 0.000000] Machine model: Renesas H3ULCB board based on r8a7795 ES2.0+
[ 0.000000] earlycon: xenboot0 at I/O port 0x0 (options '')
[ 0.000000] bootconsole [xenboot0] enabled
[ 0.000000] Xen 4.12 support found
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] Reserved memory: created CMA memory pool at
0x0000000057000000, size 400 MiB
[ 0.000000] OF: reserved mem: initialized node linux,cma@57000000,
compatible id shared-dma-pool
[ 0.000000] Reserved memory: created CMA memory pool at
0x0000000070000000, size 256 MiB
[ 0.000000] OF: reserved mem: initialized node
linux,multimedia@70000000, compatible id shared-dma-pool
[ 0.000000] cma: dma_contiguous_reserve(limit 100000000)
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem
0x0000000000000000-0x000000063fffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0x63ff90c00-0x63ff926ff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000057000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x000000063fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000057000000-0x000000007fffffff]
[ 0.000000] node 0: [mem 0x0000000500000000-0x000000053fffffff]
[ 0.000000] node 0: [mem 0x0000000600000000-0x000000063fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000057000000-0x000000063fffffff]
[ 0.000000] On node 0 totalpages: 692224
[ 0.000000] DMA zone: 2624 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 167936 pages, LIFO batch:31
[ 0.000000] Normal zone: 8192 pages used for memmap
[ 0.000000] Normal zone: 524288 pages, LIFO batch:31
[ 0.000000] bootmem alloc of 64 bytes failed!
[ 0.000000] Kernel panic - not syncing: Out of memory
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
4.14.75-ltsi-yocto-standard #3
[ 0.000000] Hardware name: Renesas H3ULCB board based on r8a7795 ES2.0+ (DT)
[ 0.000000] Call trace:
[ 0.000000] [<ffff000008089ae8>] dump_backtrace+0x0/0x3c0
[ 0.000000] [<ffff000008089ebc>] show_stack+0x14/0x20
[ 0.000000] [<ffff000008af20e8>] dump_stack+0x9c/0xbc
[ 0.000000] [<ffff0000080ce770>] panic+0x11c/0x28c
[ 0.000000] [<ffff0000090478fc>] free_bootmem_late+0x0/0x7c
[ 0.000000] [<ffff000009047d90>] __alloc_bootmem_low+0x2c/0x38
[ 0.000000] [<ffff0000090329fc>] setup_arch+0x258/0x4d8
[ 0.000000] [<ffff00000903083c>] start_kernel+0x64/0x3ac
[ 0.000000] ---[ end Kernel panic - not syncing: Out of memory
Interesting, the function __alloc_bootmem_low will try to allocate low
memory. The limit for low memory is based on the physical DMA address
limit (see ARCH_LOW_ADDRESS_LIMIT).
I guess you have CONFIG_ZONE_DMA32 (was renamed from CONFIG_ZONE_DMA is
recent Linux) set. So the DMA limit will be computed by max_zone_dma_phys().
In your setup, if I am not mistaken, the limit will be 4GB. However, you
have no available memory below 4GB because it is used for
reserved-memory. So it makes sense for __alloc_bootmem_low to fail.
In your previous setup (i.e without reserved memory), all the memory was
probably still above 4GB. In that case max_zone_dma_phys() will return
an higher "lower" limit. So __alloc_bootmem_low is going to succeed.
AFAICT, this is probably a Linux bug. Now the question is whether
__alloc_bootmem_low should be switch to alloc_bootmem or we need to fix
the implementation of the function. Note that recent Linux (5.0 and
onwards) have switch to memblock API, yet I think the issue is still here.
Fixing Linux is probably a good idea, but this means that upgrade of Xen
may break Linux boot. In one hand, reserved-memory never worked with Xen
(user have to manually disable it). So this is not a regression. On the
other hand, a user will now need to upgrade their Linux (could be
difficult if tie to a BSP kernel) or apply a fix (see above).
It might be possible to rework Dom0 memory allocation to limit the
damage or even re-order the binary in memory. Amit, could you post the
full Xen log with earlyprintk enabled?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
