On Thu, Jan 12, 2023 at 11:32:42PM +0100, Joost van Zwieten wrote:
>
>
> On Wed, Jan 11, 2023 at 19:10, Tom Rini <tr...@konsulko.com> wrote:
> > On Wed, Jan 11, 2023 at 10:08:59PM +0100, Joost van Zwieten wrote:
> > >
> > >
> > > On Wed, Jan 11, 2023 at 15:38, Tom Rini <tr...@konsulko.com> wrote:
> > > > On Wed, Jan 11, 2023 at 09:28:42PM +0100, Joost van Zwieten wrote:
> > > > >
> > > > >
> > > > > On Tue, Jan 10, 2023 at 18:29, Tom Rini <tr...@konsulko.com>
> > > wrote:
> > > > > > On Wed, Jan 11, 2023 at 12:10:42AM +0100, Joost van Zwieten
> > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jan 10, 2023 at 18:08, Tom Rini
> > > <tr...@konsulko.com>
> > > > > wrote:
> > > > > > > > On Wed, Jan 11, 2023 at 12:01:46AM +0100, Joost van
> > > Zwieten
> > > > > wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Jan 10, 2023 at 13:41, Tom Rini
> > > > > <tr...@konsulko.com>
> > > > > > > wrote:
> > > > > > > > > > On Tue, Jan 10, 2023 at 09:13:32AM +0100, Joost van
> > > > > Zwieten
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Dear maintainers,
> > > > > > > > > > >
> > > > > > > > > > > As of commit
> > > 4963f63fe61f15329d77472a762b1d8bf754d24b
> > > > > > > U-Boot
> > > > > > > > > fails
> > > > > > > > > > > to start
> > > > > > > > > > > a kernel (with `bootz`) on my Odroid U2 unless I
> > > > > force
> > > > > > > > > > > `initrd_high`, e.g.
> > > > > > > > > > > to `0x50000000`. With commit 4963f63f and
> > > > > `initrd_high`
> > > > > > > > > *unset* I
> > > > > > > > > > > get the
> > > > > > > > > > > following output:
> > > > > > > > > > >
> > > > > > > > > > > U-Boot 2020.10-rc2-00314-g4963f63fe6 (Jan 09
> > > 2023
> > > > > -
> > > > > > > 23:59:31
> > > > > > > > > > > +0100)
> > > > > > > > > > >
> > > > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > > > Type: u3
> > > > > > > > > > > DRAM: 2 GiB
> > > > > > > > > > > LDO20@VDDQ_EMMC_1.8V: set 1800000 uV;
> > > enabling
> > > > > > > > > > > LDO22@VDDQ_EMMC_2.8V: set 2800000 uV;
> > > enabling
> > > > > > > > > > > LDO21@TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > > > Loading Environment from MMC... *** Warning
> > > - bad
> > > > > CRC,
> > > > > > > using
> > > > > > > > > > > default
> > > > > > > > > > > environment
> > > > > > > > > > >
> > > > > > > > > > > In: serial
> > > > > > > > > > > Out: serial
> > > > > > > > > > > Err: serial
> > > > > > > > > > > Boot device: MMC(2)
> > > > > > > > > > > Net: No ethernet found.
> > > > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${fdt_addr_r}
> > > > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > > > 53440 bytes read in 50 ms (1 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${kernel_addr_r}
> > > > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > > > 4973056 bytes read in 182 ms (26.1 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${ramdisk_addr_r}
> > > > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > > > 22231585 bytes read in 777 ms (27.3 MiB/s)
> > > > > > > > > > > Odroid # env set bootargs
> > > console=ttySAC1,115200n8
> > > > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > > > ${fdt_addr_r}
> > > > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 -
> > > 0x4be200 ]
> > > > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > > > Loading Ramdisk to b9947000, end bae7aa21
> > > ...
> > > > > OK
> > > > > > > > > > > Loading Device Tree to b9936000, end
> > > b99460bf
> > > > > ... OK
> > > > > > > > > > >
> > > > > > > > > > > Starting kernel ...
> > > > > > > > > > >
> > > > > > > > > > > And that's all I ever see. Normally the initrd
> > > loads
> > > > > a
> > > > > > > module
> > > > > > > > > that
> > > > > > > > > > > causes an
> > > > > > > > > > > LED on the Odroid to blink, and this is not
> > > happening
> > > > > > > either,
> > > > > > > > > so
> > > > > > > > > > > I'm pretty
> > > > > > > > > > > confident the kernel doesn't start or at least
> > > > > crashes
> > > > > > > before
> > > > > > > > > > > producing
> > > > > > > > > > > output. If I set `initrd_high` to `0x50000000`
> > > (or
> > > > > > > something
> > > > > > > > > in the
> > > > > > > > > > > neighborhood) the kernel starts just fine:
> > > > > > > > > > >
> > > > > > > > > > > U-Boot 2020.10-rc2-00314-g4963f63fe6 (Jan 09
> > > 2023
> > > > > -
> > > > > > > 23:59:31
> > > > > > > > > > > +0100)
> > > > > > > > > > >
> > > > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > > > Type: u3
> > > > > > > > > > > DRAM: 2 GiB
> > > > > > > > > > > LDO20@VDDQ_EMMC_1.8V: set 1800000 uV;
> > > enabling
> > > > > > > > > > > LDO22@VDDQ_EMMC_2.8V: set 2800000 uV;
> > > enabling
> > > > > > > > > > > LDO21@TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > > > Loading Environment from MMC... *** Warning
> > > - bad
> > > > > CRC,
> > > > > > > using
> > > > > > > > > > > default
> > > > > > > > > > > environment
> > > > > > > > > > >
> > > > > > > > > > > In: serial
> > > > > > > > > > > Out: serial
> > > > > > > > > > > Err: serial
> > > > > > > > > > > Boot device: MMC(2)
> > > > > > > > > > > Net: No ethernet found.
> > > > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${fdt_addr_r}
> > > > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > > > 53440 bytes read in 49 ms (1 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${kernel_addr_r}
> > > > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > > > 4973056 bytes read in 181 ms (26.2 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${ramdisk_addr_r}
> > > > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > > > 22231585 bytes read in 777 ms (27.3 MiB/s)
> > > > > > > > > > > Odroid # env set bootargs
> > > console=ttySAC1,115200n8
> > > > > > > > > > > Odroid # env set initrd_high 0x50000000
> > > > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > > > ${fdt_addr_r}
> > > > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 -
> > > 0x4be200 ]
> > > > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > > > Loading Ramdisk to 4eacc000, end 4ffffa21
> > > ...
> > > > > OK
> > > > > > > > > > > Loading Device Tree to bae6a000, end
> > > bae7a0bf
> > > > > ... OK
> > > > > > > > > > >
> > > > > > > > > > > Starting kernel ...
> > > > > > > > > > >
> > > > > > > > > > > [ 0.000000] Booting Linux on physical CPU
> > > 0xa00
> > > > > > > > > > > <truncated>
> > > > > > > > > > >
> > > > > > > > > > > The difference between those two runs is the
> > > location
> > > > > > > where
> > > > > > > > > U-Boot
> > > > > > > > > > > loads the
> > > > > > > > > > > initrd. The parent commit of 4963f63f boots fine
> > > > > without
> > > > > > > > > setting
> > > > > > > > > > > `initrd_high`:
> > > > > > > > > > >
> > > > > > > > > > > U-Boot 2020.10-rc2-00313-gdfaf6a5797 (Jan 10
> > > 2023
> > > > > -
> > > > > > > 00:13:19
> > > > > > > > > > > +0100)
> > > > > > > > > > >
> > > > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > > > Type: u3
> > > > > > > > > > > DRAM: 2 GiB
> > > > > > > > > > > LDO20@VDDQ_EMMC_1.8V: set 1800000 uV;
> > > enabling
> > > > > > > > > > > LDO22@VDDQ_EMMC_2.8V: set 2800000 uV;
> > > enabling
> > > > > > > > > > > LDO21@TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > > > Loading Environment from MMC... *** Warning
> > > - bad
> > > > > CRC,
> > > > > > > using
> > > > > > > > > > > default
> > > > > > > > > > > environment
> > > > > > > > > > >
> > > > > > > > > > > In: serial
> > > > > > > > > > > Out: serial
> > > > > > > > > > > Err: serial
> > > > > > > > > > > Boot device: MMC(2)
> > > > > > > > > > > Net: No ethernet found.
> > > > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${fdt_addr_r}
> > > > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > > > 53440 bytes read in 49 ms (1 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${kernel_addr_r}
> > > > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > > > 4973056 bytes read in 181 ms (26.2 MiB/s)
> > > > > > > > > > > Odroid # load mmc
> > > ${mmcbootdev}:${mmcbootpart}
> > > > > > > > > ${ramdisk_addr_r}
> > > > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > > > 22231585 bytes read in 776 ms (27.3 MiB/s)
> > > > > > > > > > > Odroid # env set bootargs
> > > console=ttySAC1,115200n8
> > > > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > > > ${fdt_addr_r}
> > > > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 -
> > > 0x4be200 ]
> > > > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > > > Loading Ramdisk to 4eacc000, end 4ffffa21
> > > ...
> > > > > OK
> > > > > > > > > > > Loading Device Tree to 4eabb000, end
> > > 4eacb0bf
> > > > > ... OK
> > > > > > > > > > >
> > > > > > > > > > > Starting kernel ...
> > > > > > > > > > >
> > > > > > > > > > > [ 0.000000] Booting Linux on physical CPU
> > > 0xa00
> > > > > > > > > > > <truncated>
> > > > > > > > > > >
> > > > > > > > > > > I've tested two different kernels from Debian
> > > > > (buster and
> > > > > > > > > bullseye)
> > > > > > > > > > > and both
> > > > > > > > > > > have the same problem. I'm building U-Boot
> > > using the
> > > > > > > default
> > > > > > > > > config
> > > > > > > > > > > for
> > > > > > > > > > > board `odroid` and boot the Odroid from an SD
> > > card
> > > > > with
> > > > > > > > > firmware
> > > > > > > > > > > (`bl1`,
> > > > > > > > > > > `bl2` and `tzsw`) from Hardkernel's clone of the
> > > > > u-boot
> > > > > > > > > repository
> > > > > > > > > > > [1].
> > > > > > > > > > > There are no peripherals connected apart from
> > > the SD
> > > > > card
> > > > > > > and
> > > > > > > > > the
> > > > > > > > > > > serial
> > > > > > > > > > > console. Please let me know if you need more
> > > > > information.
> > > > > > > > > >
> > > > > > > > > > Interesting. So what does "bdi" show, both in the
> > > older
> > > > > good
> > > > > > > and
> > > > > > > > > > then current tree? Specifically the information
> > > about
> > > > > where
> > > > > > > DRAM
> > > > > > > > > is, and
> > > > > > > > > > how large it is? Thanks.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Tom
> > > > > > > > >
> > > > > > > > > Both 4963f63f (not working) and its parent (working)
> > > > > produce
> > > > > > > the
> > > > > > > > > same
> > > > > > > > > output:
> > > > > > > > >
> > > > > > > > > boot_params = 0x40000100
> > > > > > > > > DRAM bank = 0x00000000
> > > > > > > > > -> start = 0x40000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000001
> > > > > > > > > -> start = 0x50000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000002
> > > > > > > > > -> start = 0x60000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000003
> > > > > > > > > -> start = 0x70000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000004
> > > > > > > > > -> start = 0x80000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000005
> > > > > > > > > -> start = 0x90000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000006
> > > > > > > > > -> start = 0xa0000000
> > > > > > > > > -> size = 0x10000000
> > > > > > > > > DRAM bank = 0x00000007
> > > > > > > > > -> start = 0xb0000000
> > > > > > > > > -> size = 0x0ff00000
> > > > > > > > > memstart = 0x40000000
> > > > > > > > > memsize = 0x7ff00000
> > > > > > > > > flashstart = 0x00000000
> > > > > > > > > flashsize = 0x00000000
> > > > > > > > > flashoffset = 0x00000000
> > > > > > > > > baudrate = 115200 bps
> > > > > > > > > relocaddr = 0xbfe83000
> > > > > > > > > reloc off = 0x7c083000
> > > > > > > > > Build = 32-bit
> > > > > > > > > current eth = unknown
> > > > > > > > > ethaddr = (not set)
> > > > > > > > > IP addr = <NULL>
> > > > > > > > > fdt_blob = 0xbae7bf00
> > > > > > > > > new_fdt = 0xbae7bf00
> > > > > > > > > fdt_size = 0x00002fa0
> > > > > > > > > lmb_dump_all:
> > > > > > > > > memory.cnt = 0x1
> > > > > > > > > memory.size = 0x0
> > > > > > > > > memory.reg[0x0].base = 0x40000000
> > > > > > > > > .size = 0x7ff00000
> > > > > > > > >
> > > > > > > > > reserved.cnt = 0x1
> > > > > > > > > reserved.size = 0x0
> > > > > > > > > reserved.reg[0x0].base = 0xbae7acd0
> > > > > > > > > .size = 0x5085330
> > > > > > > > > arch_number = 0x000010c1
> > > > > > > > > TLB addr = 0xbfef0000
> > > > > > > > > irq_sp = 0xbae7bef0
> > > > > > > > > sp start = 0xbae7bee0
> > > > > > > > > Board Type = 0
> > > > > > > > > Early malloc usage: f0 / 400
> > > > > > > > >
> > > > > > > > > I'm not sure if this is relevant, but with the
> > > failing
> > > > > commit
> > > > > > > I can
> > > > > > > > > get the
> > > > > > > > > kernel to start with `initrd_high` set to
> > > `0x70000000` or
> > > > > > > lower,
> > > > > > > > > but not
> > > > > > > > > higher.
> > > > > > > >
> > > > > > > > OK, so 8 whole banks, of 256MB each. Does this platform
> > > have
> > > > > 2GB
> > > > > > > of
> > > > > > > > memory for real?
> > > > > > > >
> > > > > > > > --
> > > > > > > > Tom
> > > > > > >
> > > > > > > Yes, it does. See
> > > https://www.hardkernel.com/shop/odroid-u3/ .
> > > > > >
> > > > > > Thanks. I guess my current thought is that in the absence of
> > > > > adding
> > > > > > earlycon to the kernel args, to perhaps get more info out of
> > > the
> > > > > kernel
> > > > > > as to when / where it's dying, that bootm_size should be set
> > > to
> > > > > > 0x10000000 (256MB) to replicate the old behavior, on this
> > > > > platform.
> > > > > > Jaehoon, do you have any other ideas?
> > > > > >
> > > > > > --
> > > > > > Tom
> > > > >
> > > > > I was not aware of the existence of `earlycon`. Running the
> > > first
> > > > > failing
> > > > > commit 4963f63f with `earlycon` and with `initrd_high` *unset*
> > > > > reveals a
> > > > > kernel panic:
> > > > >
> > > > > Kernel image @ 0x41000000 [ 0x000000 - 0x4be200 ]
> > > > > ## Flattened Device Tree blob at 40800000
> > > > > Booting using the fdt blob at 0x40800000
> > > > > Loading Ramdisk to b9947000, end bae7aa21 ... OK
> > > > > Loading Device Tree to b9936000, end b99460bf ... OK
> > > > >
> > > > > Starting kernel ...
> > > > >
> > > > > [ 0.000000] Booting Linux on physical CPU 0xa00
> > > > > [ 0.000000] Linux version 5.10.0-20-armmp
> > > > > (debian-ker...@lists.debian.org) (gcc-10 (Debian 10.2.1-6)
> > > 10.2.1
> > > > > 20210110,
> > > > > GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian
> > > 5.10.158-2
> > > > > (2022-12-13)
> > > > > [ 0.000000] CPU: ARMv7 Processor [413fc090] revision 0
> > > > > (ARMv7),
> > > > > cr=10c5387d
> > > > > [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT
> > > > > aliasing
> > > > > instruction cache
> > > > > [ 0.000000] OF: fdt: Machine model: Hardkernel ODROID-U3
> > > > > board based
> > > > > on Exynos4412
> > > > > [ 0.000000] earlycon: exynos4210 at MMIO 0x13810000
> > > (options
> > > > > '')
> > > > > [ 0.000000] printk: bootconsole [exynos4210] enabled
> > > > > [ 0.000000] Memory policy: Data cache writealloc
> > > > > [ 0.000000] efi: UEFI not found.
> > > > > [ 0.000000] Reserved memory: created DMA memory pool at
> > > > > 0xbf700000,
> > > > > size 8 MiB
> > > > > [ 0.000000] OF: reserved mem: initialized node
> > > > > region_mfc_right,
> > > > > compatible id shared-dma-pool
> > > > > [ 0.000000] Reserved memory: created DMA memory pool at
> > > > > 0xbd300000,
> > > > > size 36 MiB
> > > > > [ 0.000000] OF: reserved mem: initialized node
> > > > > region_mfc_left,
> > > > > compatible id shared-dma-pool
> > > > > [ 0.000000] cma: Reserved 16 MiB at 0xbc000000
> > > > > [ 0.000000] Samsung CPU ID: 0xe4412220
> > > > > [ 0.000000] Zone ranges:
> > > > > [ 0.000000] DMA [mem
> > > > > 0x0000000040000000-0x000000006fffffff]
> > > > > [ 0.000000] Normal empty
> > > > > [ 0.000000] HighMem [mem
> > > > > 0x0000000070000000-0x00000000bd2fffff]
> > > >
> > > > So here we go. Initrd cannot be in highmem. I cannot find this
> > > > documented, quickly, under
> > > https://www.kernel.org/doc/html/latest/ so I
> > > > guess this is more just background knowledge than formal
> > > knowledge. This
> > > > also matches up with your noting that initrd_high=0x70000000
> > > works.
> > > >
> > > > --
> > > > Tom
> > >
> > > I found this [1]. Section 5 mentions placing the initrd in lowmem,
> > > if I
> > > understand everything correctly.
> >
> > Ah, right, there it is, I just skimmed too quickly.
> >
> > > Setting `bootm_size` to ` 0x10000000` works for me. I can write a
> > > patch that
> > > adds `bootm_size=0x100000000` to `CFG_EXTRA_ENV_SETTINGS` in
> > > `include/configs/odroid.h` if you want, but I don't understand the
> > > fix, high
> > > mem and low mem well enough to write a decent commit message. I can
> > > certainly test (a backport of, see bug reports [2], [3]) a patch.
> > >
> > > Thanks for identifying the problem and proposing a fix!
> >
> > So, bootm_size is the variable which tells U-Boot that when relocating
> > initrd/fdt around, it must be within memory start + bootm_size, as a way
> > of enforcing the low-memory requirement (or similar constraints, aarch64
> > says first 1GB of memory). So technically, could set that to 0x30000000
> > instead of 0x10000000 which is just what the implicit limit was before
> > the change in question. A patch as you suggest would be right, so yes,
> > please do and thanks again for testing / digging on the platform itself!
>
> Ok, I'll try to make a patch then! I found `CFG_SYS_BOOTMAPSZ` in the
> README. Is that one prefered over adding `bootm_size` to the default
> environment? For boards based on Exynos5 the latter is used (see
> `include/configs/exynos5-common.h`).Honestly, both are equally fine. I think I slightly prefer the #define approach only because for the environment one you have to reset to the default in order to get the changes in your running system. -- Tom
signature.asc
Description: PGP signature
