Bug#1035477: installation-reports: grub or kernel of D-I image hangs on Thinkpad X13s after "EFI stub: Exiting boot services..."

[Emanuele Rocca]

Hi Axel,

On 2023-05-03 08:16, Axel Beckert wrote:
> After choosing either "expert install" (first and prefered try) or
> "graphical expert install" (second try and second choice) I saw these
> three lines and then nothing more happened:
> 
>   EFI stub: Booting Linux Kernel...
>   EFI stub: Using DTB from configuration table
>   EFI stub: Exiting boot services...
> 
> Waited 10 minutes at least, i.e. left it alone, did something else and
> came back later. Even adding the "debug" kernel parameter didn't bring
> up any more details on what's happening.

Perhaps we can get some more output by adding 'earlycon=efifb
keep_bootcon' after 'debug' to the kernel CLI.

Additionally, please try adding 'set debug=linux' to grub's
configuration as well -- after the setparams directive for example.

See attached screenshot.

Thanks,
  Emanuele

Bug#1035477: installation-reports: grub or kernel of D-I image hangs on Thinkpad X13s after "EFI stub: Exiting boot services..."

[Emanuele Rocca]

Hi again,

On 2023-05-03 08:16, Axel Beckert wrote:
> Machine: Thinkpad X13s (ARM)

Sorry, I've realized only after sending my previous message and having a
coffee that this is about the X13s. :)

The X13s is unfortunately not supported by Linux 6.1 that Bookworm will
ship with. Hopefully 6.4 will include all the needed patches.

> But Dimitri's Ubuntu Installer Live image lunar-desktop-arm64.iso from
> https://people.canonical.com/~xnox/ubuntu-concept/full/daily-live/current/
> as of 2023-03-07 20:08 booted fine without these issues.
> 
> It would be nice if Dimitri (xnox@d.o, Cc'ed) could tell us which fix
> has been applied by him (or Ubuntu) to this image so that we can fix
> this issue also in Debian, maybe even before the release of Bookworm.

If that image works, then it includes quite a few of out of tree kernel
patches. As per [1] they are unfortunately not going to be applied to
the Debian kernel, we have to wait for upstream inclusion.

  Emanuele

[1] https://wiki.debian.org/DebianKernelPatchAcceptanceGuidelines

Bug#1035477: installation-reports: grub or kernel of D-I image hangs on Thinkpad X13s after "EFI stub: Exiting boot services..."

[Axel Beckert]

Hi Emanuele,

Emanuele Rocca wrote:
> On 2023-05-03 08:16, Axel Beckert wrote:
> > Machine: Thinkpad X13s (ARM)
> 
> Sorry, I've realized only after sending my previous message and having a
> coffee that this is about the X13s. :)

:-)

> The X13s is unfortunately not supported by Linux 6.1 that Bookworm will
> ship with.

Oh, ok. I see.

> Hopefully 6.4 will include all the needed patches.

xnox's image uses 6.2 so far, i.e. also something newer than 6.1.

> > But Dimitri's Ubuntu Installer Live image lunar-desktop-arm64.iso from
> > https://people.canonical.com/~xnox/ubuntu-concept/full/daily-live/current/
> > as of 2023-03-07 20:08 booted fine without these issues.
> > 
> > It would be nice if Dimitri (xnox@d.o, Cc'ed) could tell us which fix
> > has been applied by him (or Ubuntu) to this image so that we can fix
> > this issue also in Debian, maybe even before the release of Bookworm.
> 
> If that image works, then it includes quite a few of out of tree kernel
> patches.

I see. I kinda hoped it would just be some not enabled drivers or bugs
in the .dtb file or so as the D-I RC2 announcement mentioned the X13s
explicitly (admittelly only with flash-kernel having gotten support
for it).

> As per [1] they are unfortunately not going to be applied to
> the Debian kernel, we have to wait for upstream inclusion.

Sure. I intend to run Debian Unstable on it anyway, so I can wait. :-)

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Re: Bug#1029843: brcmfmac: requested firmware filename inconsistent with linux-firmware.git on non-devicetree systems

[Diederik de Haas]

Control: block -1 1035505

On Thursday, 4 May 2023 01:41:12 CEST Cyril Brulebois wrote:
> Diederik de Haas  (2023-05-04):
> > And that makes it a firmware-brcm80211 issue and now it all does make
> > sense as it now all does tie together :-)
> 
> Great, that's what it looked like to me but I was afraid I could have
> misunderstood the situation and I didn't want to digress in a wrong
> direction…
> 
> > So while upstream does define the link from what I earlier called
> > 'weird' name to brcmfmac43455-sdio.raspberrypi,4-model-b.txt, the Debian
> > firmware-brcm80211 package does not define that link and is therefor
> > missing.
> 
> Adding the links will at least make those paths shows up in the
> Contents-firmware indices generated by debian-cd. Those contain 3
> “columns”: path, package, component (the current format string is
> "%-55s %s %s\n").

Assuming the '55' stands for max 55 chars, that could be an issue?

> Even if hw-detect didn't misbehave because of spaces in filenames (i.e.

It turns out that spaces (and or backslashes to escape them) seems to be an 
issue for the Debian scripts used to make the Debian firmware-nonfree packages 
too. See https://bugs.debian.org/1035505 for details.
Once that is fixed, I can submit my MR to add those missing symlinks.

> For Bookworm, given we expect the firmware package to be adjusted to
> include those symlinks, what if hw-detect had some little cheatsheet,

Seems doable. I'm not going to spend time trying to make that though.
I know virtually nothing about d-i/hw-detect internals, so it seems very 
unwise for me to even try it.
Given the (specific) subject at hand, you won't be surprised that there's also 
a motivational issue ;-)

Cheers, 
 Diederik

signature.asc
Description: This is a digitally signed message part.

Bug#1035392: Bug#1029843: live-boot: Devices Requiring Firmware: multiple requested files in single line overlapping / special characters

[James Addison]

[ replying with some re-ordering ]

On Wed, 3 May 2023 at 21:23, Pete Batard  wrote:

> Obviously, with the idea of not having ARM based device that are
> constrained to a single OS (be it Windows, Linux, BSD or something> else), 
> and considering that Windows and Device Tree don't work together,
> you want to go with a mode of operation that isn't Linux specific,
> which, even if ACPI has its drawbacks, pretty much forces the use of
> ACPI over Device Tree. Else, you have Linux going into the exact
> Microsoft strong-arm tactics that it should strive to avoid...

Yep, and for those situations, that's a point in favour of the third
"System Table Selection" value that I failed to mention:
"ACPI+Devicetree".

I'm cautious about recommending it, given an understanding that
enabling ACPI could increase the amount of non-free code (ACPI Machine
Language, in particular) that may run on the system as a result.
Perhaps there could be counterbalancing functionality benefits and/or
energy-usage savings... even then I'd recommend proceeding cautiously.

I'm not sure I'm qualified to say much about Debian's compatibility
with other operating systems on the same machine, other than to
mention that I do think it's highly compatible and that that's
something that maintainers, developers and users care about.

> On 2023.05.03 17:29, James Addison wrote:
> >* Perhaps Devicetree is a better default in EDK2 for ARM systems?
> > (that wouldn't solve the root cause, though)
>
> Please note that the reason why the Raspberry Pi UEFI firmware defaults
> to ACPI is so that this ARM device follows the relatively new ARM SBBR
> standard [3], which we (hopefully) expect more and more ARM64 based
> device to follow.

Slightly off-topic: do you know of cases where ACPI has helped a
vendor to adapt to shifting operating system interfaces or achieve
significant energy-usage savings?  I think that understanding some of
those could help to begin to address gaps that Debian/Linux/other
components have.

(and to mention why I ask.. I've been reading some of the history[1],
definitions[2], rationale[3], and state-of-support[4] around
DeviceTree and ACPI in Linux, including in relation to ARM servers.
it looks like I have a decade-or-so of history to catch up on there)

[1] - 
https://lore.kernel.org/linux-arm-kernel/CAOesGMjKeRb=ffjm0mabdihbeicgm4eqw9d5i_6-rfxtnpb...@mail.gmail.com/

[2] - https://elinux.org/Device_Tree_What_It_Is

[3] - https://www.secretlab.ca/archives/151

[4] - https://www.kernel.org/doc/html/v6.3/arm64/arm-acpi.html

Re: Bug#1034650: debian-installer: bookworm d-i rc1: apt-get clean breaks bash-completion

[Cyril Brulebois]

Hi David,

David Kalnischkies  (2023-05-04):
> before I am playing bug ping-pong (as I don't think we can do much
> about this on the apt side), let me ask a d-i question first:

Entirely fair, thanks. Putting debian-boot@ in copy since currently the
bug is assigned to apt… and others might have some feedback about my
proposal(s).

> Why is d-i calling 'apt-get clean' to begin with?

Checking occurrences of `apt-get clean` (I didn't check for variations)
in our packages, I see two callers.

One is pkgsel (which deals with tasksel and pulls many/most packages),
and one later on in finish-install (which as you can guess runs at the
end).

> Is it perhaps just wanting to remove /var/cache/apt/archives/*.deb ?
> If so, could you switch to: -o APT::Keep-Downloaded-Packages=false
> (well, probably in a conf file rather than on each command line).

There are various apt-get (not clean) calls, some of them wrapped, and I
have no idea what the consequences of tweaking them could be. In any
case, while deplying a conf file during the installation process could
be attempted, it's far too late to try toying with that…

We can probably keep track of the suggestion via some bug report against
debian-installer, so that someone can investigate that post-Bookworm
(probably removing the possible tweak discussed below).

> This instructs apt(-get) to remove the deb file right after it has
> installed them with dpkg, which might even be beneficial for space
> constraint installation targets. At the very least it is a much used
> & tested option as it is the default with apt (but not apt-get).

OK, thanks.

> d-i could also "just" run 'apt-cache gencaches' to create the caches;
> but perhaps it is intended that the caches are gone (docker people e.g.
> like to do that I am told, but they nuke other things as well…).

The finish-install call can be traced back to:
  https://bugs.debian.org/586434

> > > - Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are
> > >   not present (for example, by recreating them on-the-fly)
> 
> The completion script explicitly disables the recreation on-the-fly as
> creating the files takes a while robbing users for many seconds of their
> interactivity. So, we can't just "fix" the completion script as that has
> a(nother?) set of users complain as well.

With all the backstory, the interactions/dependencies between
bash-completion and apt's internal files became clear, thanks.

For Bookworm, I'm tempted to add `apt-cache gencaches` after `apt-get
clean` in finish-install, restoring the files bash-completion relies on.

That'd waste some little time, rebuilding some of the files that were
just deleted… but at least we wouldn't risk anything in the rest of the
installation process.

(Maybe making the extra call conditional to bash-completion's being
installed? Not sure about the “small optimization” vs. “different apt
files depending on the installed packages” balance.)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1035096: installation-report Bookworm RC2 GRUB not installed

[Peter Ehlert]

On 5/2/23 11:59, Pascal Hambourg wrote:

On 02/05/2023 at 15:21, Peter Ehlert wrote:


DI asks on which drive to install GRUB
User says disk X
DI attempts to install on Drive Y
result ... GRUB does not get installed at all


This is completely different from what I understood.

The statement "GRUB does not get installed at all" is refuted by 
/var/log/syslog which indicates clearly that GRUB was successfully 
installed on /dev/sdb:


Apr 28 23:55:51 grub-installer: info: Running chroot /target 
grub-install  --force "/dev/sdb"

(...)
Apr 28 23:56:01 grub-installer: Installation finished. No error 
reported.


Can you described what happened exactly ?
Were you prompted to "Install the GRUB boot loader to your primary 
drive?" ? If yes, did you answer "yes" or "no" ?
Where you then prompted to select the "Device for boot loader 
installation" ? If yes, what option did you select exactly ?

from my initial installation-report:
"when install was finished, a list of drives was displayed, asking where 
to install GRUB.

the install drive was highlighted
I selected the drive with bios_grub
the display showed "installing GRUB on (...drive it was installed on...)"

Pardon me for not being more clear. I will elaborate from the photos I took

"Install the GRUB boot loader" menu ... 6 items
Enter device manually
/dev/sda (usb-SanDisk ...
/dev/sdb (ata-WL4000G ...
/dev/sdc (ata-WDC_WD300...
/dev/sdd (ata-WDC_WD300...
/dev/sde (ata-SanDisk_SDSSDA240G_162248447811)


I selected /dev/sde and pressed the Continue button

then the GUI progress bar Installing GRUB and I see Running 
"grub-install /dev/sdd" ...


in this case sdd was the install location, sde is the drive that has a 
bios_grub flagged partition




setting up BIOS to boot from a drive that does not have a bios_grub 
flagged partition results in:


"GPT-formatted disk.

Legacy boot not supported. Press any key to reboot."


If this is a message from the BIOS, then it is flawed. A compliant 
BIOS should not care about the partition table, even less the presence 
of any kind of partition. All a BIOS should care about is the presence 
of the "boot signature" 0x55, 0xAA at the end of the boot sector.


Yes, that is from the BIOS

running Debian, from the terminal:

peter@z820-3:~$ su -
Password:
root@z820-3:~# grub-install /dev/sda && update-grub
Installing for i386-pc platform.
grub-install: warning: this GPT partition label contains no BIOS Boot 
Partition; embedding won't be possible.
grub-install: error: embedding is not possible, but this is required for 
cross-disk install.

root@z820-3:~#

Re: Bug#1029843: brcmfmac: requested firmware filename inconsistent with linux-firmware.git on non-devicetree systems

[Cyril Brulebois]

Hi,

Diederik de Haas  (2023-05-04):
> Assuming the '55' stands for max 55 chars, that could be an issue?

That's not how format strings work. :)

That happily overflows:

$ printf "%-10s and the rest of the line\n" kibi
kibi   and the rest of the line

$ printf "%-10s and the rest of the line\n" "Diederik de Haas"
Diederik de Haas and the rest of the line

> It turns out that spaces (and or backslashes to escape them) seems to
> be an issue for the Debian scripts used to make the Debian
> firmware-nonfree packages too. See https://bugs.debian.org/1035505 for
> details. Once that is fixed, I can submit my MR to add those missing
> symlinks.

Oh, d-i isn't the only one expecting non-crazy paths! :)

> Seems doable. I'm not going to spend time trying to make that though.
> I know virtually nothing about d-i/hw-detect internals, so it seems
> very unwise for me to even try it.
> Given the (specific) subject at hand, you won't be surprised that
> there's also a motivational issue ;-)

I was merely writing it out so that it could be sanity-checked, I wasn't
suggesting you would be the one to implement that. (Instead, I was
implicitly volunteering myself…)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1035096: installation-report Bookworm RC2 GRUB not installed

[Pascal Hambourg]

On 04/05/2023 at 16:21, Peter Ehlert wrote:


"Install the GRUB boot loader" menu ... 6 items
Enter device manually
/dev/sda (usb-SanDisk ...
/dev/sdb (ata-WL4000G ...
/dev/sdc (ata-WDC_WD300...
/dev/sdd (ata-WDC_WD300...
/dev/sde (ata-SanDisk_SDSSDA240G_162248447811)


I selected /dev/sde and pressed the Continue button

then the GUI progress bar Installing GRUB and I see Running 
"grub-install /dev/sdd" ...


in this case sdd was the install location, sde is the drive that has a 
bios_grub flagged partition


OK, I think I managed to reproduce the issue and it shares the root 
cause with another bug I am currently chasing (GRUB is not installed at 
all). Some logic in grub-installer seems to be flawed but I still need 
to understand the original intent before I can submit a patch for both bugs.


Meanwhile, a workaround is to enter the boot device manually.

setting up BIOS to boot from a drive that does not have a bios_grub 
flagged partition results in:


"GPT-formatted disk.

Legacy boot not supported. Press any key to reboot."


If this is a message from the BIOS, then it is flawed. A compliant 
BIOS should not care about the partition table, even less the presence 
of any kind of partition. All a BIOS should care about is the presence 
of the "boot signature" 0x55, 0xAA at the end of the boot sector.


Yes, that is from the BIOS

(...)

root@z820-3:~# grub-install /dev/sda && update-grub
Installing for i386-pc platform.
grub-install: warning: this GPT partition label contains no BIOS Boot 
Partition; embedding won't be possible.
grub-install: error: embedding is not possible, but this is required for 
cross-disk install.


This failure has nothing to do with the above. You are trying to install 
GRUB on a GPT disk which is not the one containing /boot ("cross-disk"), 
and this requires a BIOS boot partition.

Bug#1035096: installation-report Bookworm RC2 GRUB not installed

[Peter Ehlert]

On May 4, 2023 3:35:40 PM Pascal Hambourg  wrote:


On 04/05/2023 at 16:21, Peter Ehlert wrote:


"Install the GRUB boot loader" menu ... 6 items
Enter device manually
/dev/sda (usb-SanDisk ...
/dev/sdb (ata-WL4000G ...
/dev/sdc (ata-WDC_WD300...
/dev/sdd (ata-WDC_WD300...
/dev/sde (ata-SanDisk_SDSSDA240G_162248447811)


I selected /dev/sde and pressed the Continue button

then the GUI progress bar Installing GRUB and I see Running
"grub-install /dev/sdd" ...

in this case sdd was the install location, sde is the drive that has a
bios_grub flagged partition


OK, I think I managed to reproduce the issue and it shares the root
cause with another bug I am currently chasing (GRUB is not installed at
all). Some logic in grub-installer seems to be flawed but I still need
to understand the original intent before I can submit a patch for both bugs.

Meanwhile, a workaround is to enter the boot device manually.

Thanks. Best of luck. I appreciate your assistance




setting up BIOS to boot from a drive that does not have a bios_grub
flagged partition results in:

"GPT-formatted disk.

Legacy boot not supported. Press any key to reboot."


If this is a message from the BIOS, then it is flawed. A compliant
BIOS should not care about the partition table, even less the presence
of any kind of partition. All a BIOS should care about is the presence
of the "boot signature" 0x55, 0xAA at the end of the boot sector.


Yes, that is from the BIOS

(...)

root@z820-3:~# grub-install /dev/sda && update-grub
Installing for i386-pc platform.
grub-install: warning: this GPT partition label contains no BIOS Boot
Partition; embedding won't be possible.
grub-install: error: embedding is not possible, but this is required for
cross-disk install.


This failure has nothing to do with the above. You are trying to install
GRUB on a GPT disk which is not the one containing /boot ("cross-disk"),
and this requires a BIOS boot partition.

ARM64 platform, very serious, UEFI hardware damage problem

[gugudu]

Hello, Debian Developers. 

I
 installed Debian 12 BooksWorm on Phytium（飞腾）'s computer. 
These PCs are based on the ARM64 UEFI platform.After restarting, The 
motherboard cannot boot and keeps black screen.The motherboard has to be 
returned to the factory for repair to solve the problem.These UEFI firmware are 
made by 
Kunlun Tech（昆仑太科） or Byo software（百敖软件）.After Debian
 operating system is installed, Debian installer will write Debian boot entries 
to the UEFI.However, the UEFI of the Phytium（飞腾）motherboard may have some kind 
of incompatibility with the installer.This write operation can cause UEFI 
damage.Some motherboard manufacturers have stopped maintaining the UEFI,so I 
would like to ask your advices on how to disable the installer writing boot 
entry to the UEFI.
Some users with similar experiences have speculated on the reason for the 
damage.
https://gitee.com/atzlinux/debian-cn/issues/I4RJDW
The uefi boot entry should be on the nvram, and BIOS discharges should be 
useless.
I guess it's a bug in uefi. When writing boot items to nvram, it breaks other 
places. And UEFI is damagedlinux indirectly let uefi write data through uefi's 
runtime interface. It probably looks like this.Why would installing the system 
will damage the firmware? Could efibootmgr write variable to damage the 
firmware?
Oh, that's possible, because /sys/firmware/efi/efivars/ is what the kernel 
reads out of the firmware for efibootmgr to use
I don't think debian should be blamed for this. Even if debian calls the 
runtime interface in the wrong position,runtime can't write itself 
badly.efibootmgr installation of new boot entries and adjusting the order are 
handled by the kerneland firmware runtime to complete the dealThat's because 
the kernel and firmware calls to runtime are not handled properly
If you are debugging UEFI, all I can think of is to add printing to all the 
functions of the EFI_RUNTIME_SERVICES interfaceto see what interfaces the 
debian installer has tuned and what parameters it has passed. The focus is on 
SetXxx functionssuch as SetVariable(), SetVirutalAddressMap and other 
functions.I have also installed ubuntu 21.04 on d2000-8 and everything works 
fine. Coexisting with the original debian 11.
My current practice is to go to the last step of the debian installation, when 
it says to remove the installation media,just power off, unplug the USB drive 
and then manually create boot entries in the bios, and then it's fine.