Bug#1006800: debian-installer: kernel mismatch for bookworm and sid installer. New release needed?

[Roland Clobus]

Package: debian-installer
Version: 20210731+deb11u2
Severity: normal

Dear maintainer(s) of debian-installer,

I've been working on reproducible live ISO images for some time now [1] and the
images can be generated in a reproducible manner.
As the next step, I want to test the functionality of the live ISO images and
recently I started working with Philip Hands to get them being tested in
openQA.

I have noticed that the officially released version debian-installer [2][3]
will not work for bookworm and sid, because the kernel version in the debian-
installer does not match the current kernel version. You recently fixed this in
git [4].

Could you release a new version of debian-installer for bookworm and sid?
Or do you recommend a different (release) strategy?
I'm aware of the daily images [5], but they are currently not being
snapshotted, which makes it impossible to reproduce an image after the older
images have been removed from [5].

With kind regards,
Roland Clobus

[1] https://wiki.debian.org/ReproducibleInstalls/LiveImages
[2]
https://snapshot.debian.org/archive/debian/20220305T025031Z/dists/sid/main/installer-
amd64/20210731/
[3] https://deb.debian.org/debian/dists/sid/main/installer-amd64/current/
[4] https://salsa.debian.org/installer-team/debian-
installer/-/blob/f810235e642e7ed266cc6a41b8fccd864180714a
[5] https://d-i.debian.org/daily-images/amd64/daily/


-- System Information:
Debian Release: bookworm/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'testing-debug'), (50, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.16.0-3-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_GB:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Bug#1006800: debian-installer: kernel mismatch for bookworm and sid installer. New release needed?

[Cyril Brulebois]

Hi,

Roland Clobus  (2022-03-05):
> I have noticed that the officially released version debian-installer
> [2][3] will not work for bookworm and sid, because the kernel version
> in the debian- installer does not match the current kernel version.
> You recently fixed this in git [4].
> 
> Could you release a new version of debian-installer for bookworm and
> sid?

We could, and should, release a new d-i and possibly an Alpha 1 at some
point, but I don't have a specific timeline for that.

> Or do you recommend a different (release) strategy?

A new debian-installer upload (prelude to the aforementioned Alpha 1) is
only going to help until src:linux gets a new ABI bump, so that's only
going to be temporary anyway.

> I'm aware of the daily images [5], but they are currently not being
> snapshotted, which makes it impossible to reproduce an image after the
> older images have been removed from [5].

If you're using a specific build, you could mirror it on your side, and
then have a way to point at the mirrored copy so that you wouldn't
depend on d-i.d.o's contents (that's an approach seen in various
projects, e.g.  time-based snapshots and tagged snapshots in Tails, even
if that's for Debian as a whole, not d-i)?

How long do you need to go back / how long do you need to keep a given
build? Maybe we could just keep (some) builds for a longer while there,
but that's at 90 days already.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1006800: debian-installer: kernel mismatch for bookworm and sid installer. New release needed?

[Roland Clobus]

+mailing list rb-general

Hi,

On 05/03/2022 12:40, Cyril Brulebois wrote:

Roland Clobus  (2022-03-05):

I have noticed that the officially released version debian-installer
[2][3] will not work for bookworm and sid, because the kernel version
in the debian- installer does not match the current kernel version.
You recently fixed this in git [4].

Could you release a new version of debian-installer for bookworm and
sid?


We could, and should, release a new d-i and possibly an Alpha 1 at some
point, but I don't have a specific timeline for that.


Understood. I assume that an Alpha 1 release will be made somewhere near 
the release date of bookworm.



Or do you recommend a different (release) strategy?


A new debian-installer upload (prelude to the aforementioned Alpha 1) is
only going to help until src:linux gets a new ABI bump, so that's only
going to be temporary anyway.


Indeed. A new debian-installer upload would need to happen in lock-step 
with every new ABI in src:linux, to guarantee a consistent state of d-i.

This could mean quite some work on your side.


I'm aware of the daily images [5], but they are currently not being
snapshotted, which makes it impossible to reproduce an image after the
older images have been removed from [5].


If you're using a specific build, you could mirror it on your side, and
then have a way to point at the mirrored copy so that you wouldn't
depend on d-i.d.o's contents (that's an approach seen in various
projects, e.g.  time-based snapshots and tagged snapshots in Tails, even
if that's for Debian as a whole, not d-i)?


I do not know how much work it is to release a new version of 
debian-installer. Currently the state of the official repository 
(deb.debian.org) is a non-working installer for bookworm and sid.


I'm looking at possible solutions here (that's why I've added the 
rb-general mailing list):

* (Manually) do official releases of debian-installer more often
  (as I wrote, openQA will soon have some tests that detect when the 
kernel version got out-of-sync)

* Automatically release git snapshots to deb.d.o instead of d-i.d.o
* Extend snapshot.d.o and/or snapshot.notset.fr to cover d-i.d.o in 
addition to deb.d.o
* No changes, and accept that older images cannot be recreated (this 
option is not preferred by me)

* Other ...


How long do you need to go back / how long do you need to keep a given
build? Maybe we could just keep (some) builds for a longer while there,
but that's at 90 days already.


Looking at https://d-i.debian.org/daily-images/amd64/, the current 
history I can see is about 15 days.


While investigating reproducible issues I personally tend to pick some 
timestamp and work on that for a longer period of time. 90 days would 
suffice completely for my purpose.


With kind regards,
Roland Clobus


OpenPGP_signature
Description: OpenPGP digital signature

Bug#1006809: Installation of sid successfully at AMD Desktop PC

[Bernhard]

Package: installation-reports

Boot method: Network PXE Boot
Image version: PXE Boot with daily:

> https://d-i.debian.org/daily-images/amd64/daily/netboot/debian-installer/amd64/initrd.gz
> https://d-i.debian.org/daily-images/amd64/daily/netboot/debian-installer/amd64/linux

Date: 2022-03-05

Machine: Self-made Desktop PC
Processor: AMD A10-5700 APU with Radeon(tm) HD Graphics
Memory: 4GB
Partitions:

> DateisystemTyp  1K-Blöcke  Benutzt Verfügbar Verw% Eingehängt auf
> udev   devtmpfs   18920640   18920640% /dev
> tmpfs  tmpfs   383788  9123828761% /run
> /dev/sda1  ext4 113809604 10202520  97779708   10% /
> tmpfs  tmpfs  191892811380   19075481% /dev/shm
> tmpfs  tmpfs 51204  51161% /run/lock
> tmpfs  tmpfs   383784   763837081% /run/user/1000

Output of lspci -knn:

> 00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h 
> (Models 10h-1fh) Processor Root Complex [1022:1410]
>   Subsystem: ASUSTeK Computer Inc. Family 15h (Models 10h-1fh) Processor 
> Root Complex [1043:8526]
> 00:01.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. 
> [AMD/ATI] Trinity [Radeon HD 7660D] [1002:9901]
>   Subsystem: ASUSTeK Computer Inc. Trinity [Radeon HD 7660D] [1043:8526]
>   Kernel driver in use: radeon
>   Kernel modules: radeon, amdgpu
> 00:01.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Trinity 
> HDMI Audio Controller [1002:9902]
>   Subsystem: ASUSTeK Computer Inc. Trinity HDMI Audio Controller 
> [1043:8526]
>   Kernel driver in use: snd_hda_intel
>   Kernel modules: snd_hda_intel
> 00:10.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> XHCI Controller [1022:7812] (rev 03)
>   Subsystem: ASUSTeK Computer Inc. FCH USB XHCI Controller [1043:8527]
>   Kernel driver in use: xhci_hcd
>   Kernel modules: xhci_pci
> 00:10.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> XHCI Controller [1022:7812] (rev 03)
>   Subsystem: ASUSTeK Computer Inc. FCH USB XHCI Controller [1043:8527]
>   Kernel driver in use: xhci_hcd
>   Kernel modules: xhci_pci
> 00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA 
> Controller [AHCI mode] [1022:7801] (rev 40)
>   Subsystem: ASUSTeK Computer Inc. FCH SATA Controller [AHCI mode] 
> [1043:8527]
>   Kernel driver in use: ahci
>   Kernel modules: ahci
> 00:12.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> OHCI Controller [1022:7807] (rev 11)
>   Subsystem: ASUSTeK Computer Inc. FCH USB OHCI Controller [1043:8527]
>   Kernel driver in use: ohci-pci
>   Kernel modules: ohci_pci
> 00:12.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> EHCI Controller [1022:7808] (rev 11)
>   Subsystem: ASUSTeK Computer Inc. FCH USB EHCI Controller [1043:8527]
>   Kernel driver in use: ehci-pci
>   Kernel modules: ehci_pci
> 00:13.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> OHCI Controller [1022:7807] (rev 11)
>   Subsystem: ASUSTeK Computer Inc. FCH USB OHCI Controller [1043:8527]
>   Kernel driver in use: ohci-pci
>   Kernel modules: ohci_pci
> 00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] FCH USB 
> EHCI Controller [1022:7808] (rev 11)
>   Subsystem: ASUSTeK Computer Inc. FCH USB EHCI Controller [1043:8527]
>   Kernel driver in use: ehci-pci
>   Kernel modules: ehci_pci
> 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller 
> [1022:780b] (rev 14)
>   Subsystem: ASUSTeK Computer Inc. FCH SMBus Controller [1043:8527]
>   Kernel driver in use: piix4_smbus
>   Kernel modules: i2c_piix4, sp5100_tco
> 00:14.2 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] FCH Azalia 
> Controller [1022:780d] (rev 01)
>   Subsystem: ASUSTeK Computer Inc. F2A85-M Series [1043:8444]
>   Kernel driver in use: snd_hda_intel
>   Kernel modules: snd_hda_intel
> 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge 
> [1022:780e] (rev 11)
>   Subsystem: ASUSTeK Computer Inc. FCH LPC Bridge [1043:8527]
> 00:14.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] FCH PCI Bridge 
> [1022:780f] (rev 40)
> 00:15.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Hudson PCI to 
> PCI bridge (PCIE port 0) [1022:43a0]
>   Kernel driver in use: pcieport
> 00:15.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Hudson PCI to 
> PCI bridge (PCIE port 1) [1022:43a1]
>   Kernel driver in use: pcieport
> 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h 
> (Models 10h-1fh) Processor Function 0 [1022:1400]
> 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h 
> (Models 10h-1fh) Processor Function 1 [1022:1401]
> 00:18.2 Host bridge [0600]: Advanced M

Bug#1006800: debian-installer: kernel mismatch for bookworm and sid installer. New release needed?

[Cyril Brulebois]

Roland Clobus  (2022-03-05):
> On 05/03/2022 12:40, Cyril Brulebois wrote:
> > We could, and should, release a new d-i and possibly an Alpha 1 at some
> > point, but I don't have a specific timeline for that.
> 
> Understood. I assume that an Alpha 1 release will be made somewhere
> near the release date of bookworm.

In the past I've tried to have an Alpha 1 released after a few months
into the new release cycle, then aim for something like a release every
1-2 months.

But the archive can disagree from time to time, and lately, I'm rather
busy with other things…

> Indeed. A new debian-installer upload would need to happen in
> lock-step with every new ABI in src:linux, to guarantee a consistent
> state of d-i. This could mean quite some work on your side.

Uploading more often could be doable; the current approach has been to
upload whenever we were getting close to wanting a new release of the
installer, following up with 1-2 more uploads if things didn't work out
immediately.

> I'm looking at possible solutions here (that's why I've added the
> rb-general mailing list):
> * (Manually) do official releases of debian-installer more often

Doable, as stated above.

>   (as I wrote, openQA will soon have some tests that detect when the
>   kernel version got out-of-sync)

Well, I do follow kernel uploads and ABI bumps closely, so that's a
problem that's already solved (git says since 2013).

FWIW we're seeing mutiple hours to multiple days of delay for some
architectures (we need the *-signed packages to get processed).

> * Automatically release git snapshots to deb.d.o instead of d-i.d.o

Uploading untested stuff to the archive, automatically? No, thanks.

> * Extend snapshot.d.o and/or snapshot.notset.fr to cover d-i.d.o in
>   addition to deb.d.o

Maybe avoidable if we go for more frequent uploads.

> * No changes, and accept that older images cannot be recreated (this
>   option is not preferred by me)
> * Other ...
> 
> > How long do you need to go back / how long do you need to keep a given
> > build? Maybe we could just keep (some) builds for a longer while there,
> > but that's at 90 days already.
> 
> Looking at https://d-i.debian.org/daily-images/amd64/, the current
> history I can see is about 15 days.

Right, ISTR that was around 30 days, glanced at the graph before hitting
the sack and didn't check thoroughly: we indeed clean more frequently
than showed on the graph.

> While investigating reproducible issues I personally tend to pick some
> timestamp and work on that for a longer period of time. 90 days would
> suffice completely for my purpose.

At the moment, daily-images is 42G, 6.8G of it being amd64. If we
estimate a ×6 in size for just that architecture, that's nearly doubling
that size, and it's only covering that one architecture. The machine is
shared across various services, but has 121G free at the moment, so we
could probably try something like that (maybe some middleground like 45
days for starters) if that helped in the very short term? Or would you
need to look at more architectures immediately?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

installation-guide_20220129~deb11u1_source.changes ACCEPTED into proposed-updates->stable-new, proposed-updates

[Debian FTP Masters]

Accepted:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Format: 1.8
Date: Sun, 06 Feb 2022 01:36:22 +0100
Source: installation-guide
Architecture: source
Version: 20220129~deb11u1
Distribution: bullseye
Urgency: medium
Maintainer: Debian Install System Team 
Changed-By: Samuel Thibault 
Changes:
 installation-guide (20220129~deb11u1) bullseye; urgency=medium
 .
   * Backport documentation fixes to bullseye.
Checksums-Sha1:
 f9f8a3cc8759b6b1bf5147e23c48185df01c6ca3 2875 
installation-guide_20220129~deb11u1.dsc
 92a8796068aba186779eb75eb42339472a88b541 3781380 
installation-guide_20220129~deb11u1.tar.xz
 12f59e015cb65fc1800ee0e0225c2ce45039a7f7 14900 
installation-guide_20220129~deb11u1_amd64.buildinfo
Checksums-Sha256:
 edf131e6df0935db99a2b293c51efa9c8ba6d41faa6c20f3c0f1a06601338078 2875 
installation-guide_20220129~deb11u1.dsc
 c50b190da824c6544b1e8affe7624c5de0b76385813d265298f294920add7ad1 3781380 
installation-guide_20220129~deb11u1.tar.xz
 e26a543bb61e84601ae3da6890393ee81570733cf80aa8451dca8ff84ca31904 14900 
installation-guide_20220129~deb11u1_amd64.buildinfo
Files:
 0f1d87dba8166b7af03df2823d756958 2875 doc optional 
installation-guide_20220129~deb11u1.dsc
 cc052acb32d139d497269576301c5745 3781380 doc optional 
installation-guide_20220129~deb11u1.tar.xz
 de762be9e445a98c57afd6981d52e60a 14900 doc optional 
installation-guide_20220129~deb11u1_amd64.buildinfo

-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEi6MnFvk67auaclLJ5pG0tXV4H2IFAmH/Iw8ACgkQ5pG0tXV4
H2I00BAAsRdVZFalacYasG+zJDlz2IkWcZGe1xIBaGabuFLtKVhvxRDxz8KRCPfI
qC0/YoFD0aqzUzYwRGvJaEdIRQGU0znIggt0Va6mIQujVhh+bEdGY57DGhlh7dqY
I3usGW0TPmQT749CaVEpUjw8L/quN3/Fw4b2P+zu9U/jvACYCPeVOnD0JK7GQYdL
+faYUlxSJ1XaJ5APuph9535HRKIv+3ZwZoWOOQf/Td0M/vY4iX16HmmeYaMGmHp8
MI2niNu4l0/hjx/fmkFLegBLI28OouQd2eIr95LoftqoiBeIXDQHCyOXHHf9er8E
vtc9ukJLCww7QXlByFl3ccCE4GN18YOcpE280PduoNFmZqHnfAq2L7fJDemDgoZG
xFqbU+ayeGYERZCGVB+m7GBGK3xG8MQpBPmCEzZXGcwU8UjbCYigGZrfrNUE1InS
BKauVxcRngmkV/6Fi8mhZsBFiulwF5gwNygfKI7KqKPuU6hBAQmYOImZOhH4UHnN
YjlxQVvu5Eveze7Vi9DTcWVqu89xGSb7B+MKFBew7wBVg1JHs3v7OrrUJX8B9Rn9
cRS9q0eJn2qsANBuVOBlyt8BKUyCXW32LrU/R1Fa7un5ftzljYwAO8m8mKflLbfg
Ip3yS0PFYLeERs5xtc8VGTO4Imk1WDokhSVPNRgZsGhzI+uG9OE=
=D4kA
-END PGP SIGNATURE-


Thank you for your contribution to Debian.