Re: Firmware GR result - what happens next?

[Santiago Ruano Rincón]

El 14/10/22 a las 13:32, Paul Wise escribió:
> On Thu, 2022-10-13 at 17:35 +0200, Julien Cristau wrote:
> 
> > I'd prefer if we could make things work vs making things fail,
> > however loudly.
> 
> There seem to be a few ways to deal with this transition:
> 
> 1. Document it in the release notes and let users handle it. This means
> lots of users won't get security updates for firmware (which are mostly
> only issued for x86 CPU microcode), since lots of folks won't read the
> release notes. This also means lots of support requests when users
> can't find the firmware package they wanted.
> 
> 2. Add some code somewhere to automatically modify the apt sources,
> somehow ensure that code is run by all Debian users and hope that other
> automated processes (like ansible/puppet) don't overwrite those changes
> and hope that users aren't storing apt sources config in packages,
> which would mean conffile prompts after the modification happens.
> 
> 3. Add some code to apt to download non-free-firmware when non-free is
> available in the sources and the downloaded Release files. This would
> solve the issue for Debian and all other derivatives too, if they
> decide to add a new non-free-firmware component too. This might not
> be accepted by apt developers as it is kind of a hack to special-case
> Debian component semantics in apt, although maybe a component mapping
> config option would be accepted. This might result in extra Debian
> support requests when users notice the new component in `apt update`. 
> This might not work for users of tools not based on apt, like cupt?
> This wouldn't result in users without non-free getting any non-free
> firmware though, which maybe we want since it is the new default?
> 
> 4. Keep all non-free-firmware packages in non-free too. This would be
> backwards compatible, but may expose bugs in dak, debian-cd, apt and
> other tools, so IIRC this has been vetoed by the archive and CD teams.
> This also wouldn't result in users without non-free getting any of the
> non-free firmware, which maybe we want since it is the new default?
> 
> Personally I would choose 4 first, I expect any potential issues could
> be easily fixed before the freeze. Next I would choose 3. Next I would
> choose 1 because I think /etc belongs to the sysadmin not packages.

5. transitional packages along with a helper package (that fails or
success during install) to prompt the user so they add non-free-firmware
section when needed.

Is there any reason why you are not considering 5.?

Cheers!

 -- Santiago


signature.asc
Description: PGP signature

Re: Firmware GR result - what happens next?

[Peter Pentchev]

On Fri, Oct 14, 2022 at 10:52:01AM +0200, Santiago Ruano Rincón wrote:
> El 14/10/22 a las 13:32, Paul Wise escribió:
> > On Thu, 2022-10-13 at 17:35 +0200, Julien Cristau wrote:
> > 
> > > I'd prefer if we could make things work vs making things fail,
> > > however loudly.
> > 
> > There seem to be a few ways to deal with this transition:
> > 
> > 1. Document it in the release notes and let users handle it. This means
> > lots of users won't get security updates for firmware (which are mostly
> > only issued for x86 CPU microcode), since lots of folks won't read the
> > release notes. This also means lots of support requests when users
> > can't find the firmware package they wanted.
> > 
> > 2. Add some code somewhere to automatically modify the apt sources,
> > somehow ensure that code is run by all Debian users and hope that other
> > automated processes (like ansible/puppet) don't overwrite those changes
> > and hope that users aren't storing apt sources config in packages,
> > which would mean conffile prompts after the modification happens.
> > 
> > 3. Add some code to apt to download non-free-firmware when non-free is
> > available in the sources and the downloaded Release files. This would
> > solve the issue for Debian and all other derivatives too, if they
> > decide to add a new non-free-firmware component too. This might not
> > be accepted by apt developers as it is kind of a hack to special-case
> > Debian component semantics in apt, although maybe a component mapping
> > config option would be accepted. This might result in extra Debian
> > support requests when users notice the new component in `apt update`. 
> > This might not work for users of tools not based on apt, like cupt?
> > This wouldn't result in users without non-free getting any non-free
> > firmware though, which maybe we want since it is the new default?
> > 
> > 4. Keep all non-free-firmware packages in non-free too. This would be
> > backwards compatible, but may expose bugs in dak, debian-cd, apt and
> > other tools, so IIRC this has been vetoed by the archive and CD teams.
> > This also wouldn't result in users without non-free getting any of the
> > non-free firmware, which maybe we want since it is the new default?
> > 
> > Personally I would choose 4 first, I expect any potential issues could
> > be easily fixed before the freeze. Next I would choose 3. Next I would
> > choose 1 because I think /etc belongs to the sysadmin not packages.
> 
> 5. transitional packages along with a helper package (that fails or
> success during install) to prompt the user so they add non-free-firmware
> section when needed.
> 
> Is there any reason why you are not considering 5.?

Do you mean having packages with the same names, but different versions
(even if only Debian revisions) and totally different contents, and also
built from different source packages, in different sections of the same
suite in the archive?... I'm... I'm not sure this would work... I think that
others already expressed some doubts as to how dak would handle that.

G'luck,
Peter

-- 
Peter Pentchev  r...@ringlet.net r...@debian.org p...@storpool.com
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13


signature.asc
Description: PGP signature

Re: Firmware GR result - what happens next?

[Marvin Renich]

* Paul Wise  [221014 01:35]:
> On Thu, 2022-10-13 at 17:35 +0200, Julien Cristau wrote:
> 
> > I'd prefer if we could make things work vs making things fail,
> > however loudly.
> 
> There seem to be a few ways to deal with this transition:
> 
> 2. Add some code somewhere to automatically modify the apt sources,
> somehow ensure that code is run by all Debian users and hope that other
> automated processes (like ansible/puppet) don't overwrite those changes
> and hope that users aren't storing apt sources config in packages,
> which would mean conffile prompts after the modification happens.

Actually, I think this would work really well.  Do not modify any
existing sources.list; drop a new file in sources.list.d.  This file can
have a default name that includes "firmware-nonfree" so is highly
unlikely to exist, but if it does, add "-NNN" suffix with the smallest
numeric NNN that does not exist.

Of course, the file would only be added if the current sources do not
include that component, which would _really_ decrease the probability of
a file name conflict to be about the same as the probability of the
earth being destroyed by an asteroid.

...Marvin

Processing of debootstrap_1.0.128_source.changes

[Debian FTP Masters]

debootstrap_1.0.128_source.changes uploaded successfully to localhost
along with the files:
  debootstrap_1.0.128.dsc
  debootstrap_1.0.128.tar.gz
  debootstrap_1.0.128_source.buildinfo

Greetings,

Your Debian queue daemon (running on host usper.debian.org)

Re: Firmware GR result - what happens next?

[Holger Levsen]

On Fri, Oct 14, 2022 at 01:32:41PM +0800, Paul Wise wrote:
> 4. Keep all non-free-firmware packages in non-free too. This would be
> backwards compatible, but may expose bugs in dak, debian-cd, apt and
> other tools, so IIRC this has been vetoed by the archive and CD teams.
> This also wouldn't result in users without non-free getting any of the
> non-free firmware, which maybe we want since it is the new default?
> 
> Personally I would choose 4 first, I expect any potential issues could
> be easily fixed before the freeze. 

but what would you then propose after the release, where we would have
exactly the same situation as we would have now. carry this on forever?


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

When this virus is over, I still want some of you all to stay away from me.


signature.asc
Description: PGP signature

debootstrap_1.0.128_source.changes ACCEPTED into unstable

[Debian FTP Masters]

Accepted:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Format: 1.8
Date: Fri, 14 Oct 2022 12:31:04 +0100
Source: debootstrap
Built-For-Profiles: noudeb
Architecture: source
Version: 1.0.128
Distribution: unstable
Urgency: low
Maintainer: Debian Install System Team 
Changed-By: Dimitri John Ledkov 
Launchpad-Bugs-Fixed: 1990856
Changes:
 debootstrap (1.0.128) unstable; urgency=low
 .
   [ Samuel Thibault ]
   * Make gbp tag produce the right tag format
 .
   [ Carsten Schoenert ]
   * Add (PureOS) byzantium as a symlink to amber
   * Add (PureOS) crimson as a symlink to amber
 .
   [ Daniel Watkins ]
   * Support Packages files which are not ordered alphabetically by Package 
field
 (LP: #1990856)
 .
   [ Dimitri John Ledkov ]
   * d/tests/unsorted-packages-files: cleanup temp files and daemon
 .
   [ Tianon Gravi ]
   * Apply "EXCLUDE_DEPENDENCY" during "resolve_deps"
Checksums-Sha1:
 51930a16183b28ad4a668ece65c857c10daa 1970 debootstrap_1.0.128.dsc
 1752388a7ebdc21625da6fe07f83bc282356d486 84339 debootstrap_1.0.128.tar.gz
 15897ebf050e9caebc40368c75d12c4c50da9a73 8632 
debootstrap_1.0.128_source.buildinfo
Checksums-Sha256:
 9e822f73c3afceccab09f1dcbe74b2506dbbd53a8a816581c4709a96790f7182 1970 
debootstrap_1.0.128.dsc
 09e7f8795fee894b77994213ee3a588e9d8f96ddf5f93afdec91e9a137aa7866 84339 
debootstrap_1.0.128.tar.gz
 7bc9f70bc7fdf496a5af6178509d448de56835e766284eb1c36d9a7380e6780a 8632 
debootstrap_1.0.128_source.buildinfo
Files:
 ce3650946ac771af2e7f517fdb736c08 1970 admin optional debootstrap_1.0.128.dsc
 da45c1b4fd4008ac96b3bf1cc67f0a34 84339 admin optional 
debootstrap_1.0.128.tar.gz
 52cf4e474dbf530e83935aabeca2cd92 8632 admin optional 
debootstrap_1.0.128_source.buildinfo

-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEE7iQKBSojGtiSWEHXm47ISdXvcO0FAmNJSKYACgkQm47ISdXv
cO07Ag/+K6Zo0nQDVenszwyJoJSYDayupG0SA4j3iA0YRO99/cTTzMhuZ3RZPN1t
O0bnQDvYJDcEnLqUiJ4KFQj/2wG2x2SK/rWtrKm9H/0GXPYVZX5Nypd24D1spGKV
PCQAu55TURFKNH0xjS0Aro0MbPtihN6lQK+GGISOmJUlONAqDD/bIFnDNyG+LlzT
V66g1nqqWk2lhR6ODHi0MfbIJGcO5i8lwXahordryxt/7pSTD1DYa6mGhMxpd/qT
9/tjeU29TKwyRoN3RI5FwT74scuicTPkyKfTJQ7fjxDmQOjPaY9n4JwbdiYiHqTu
HUQGxqxyR9cibL0ugfExpD14z9Tlu+0O84JdBVZjcyOInS7V1dTHHM89sbJ+G/3O
jr9FrlmA5OF57h3ompcmb3gngfW8ZKTS5+vfV2PNzqNX3YNmmu6ZhrH8lkQ99P9J
V546XN/617xPqAs7IxCb1S54fcI1iMgJqV+BczoQD0f4+Z32KnOQrf1vDBhSLHf3
XmzTMOY/m4r9PVTucjXHwWOx/Klihh1BGs4qCks4nRA1Ji4OlIbJkLRkFN7r7SLb
JXIEgS8NSvSQdjJ/kiCq4yzjszEUcUkYI0Pm2JTD6GqnRq2pjTXay6Tlw58QjZP2
RfY3Hj667yornsKAynVFStck8Yu3qfQgJX9/8mwhP5weWiFStFU=
=lrOE
-END PGP SIGNATURE-


Thank you for your contribution to Debian.

Re: Firmware GR result - what happens next?

[David Kalnischkies]

On Fri, Oct 14, 2022 at 01:32:41PM +0800, Paul Wise wrote:
> On Thu, 2022-10-13 at 17:35 +0200, Julien Cristau wrote:
> > I'd prefer if we could make things work vs making things fail,
> > however loudly.
> 
> There seem to be a few ways to deal with this transition:

(quotes reordered in Pauls preference order mentioned at the end)

> 4. Keep all non-free-firmware packages in non-free too. This would be
> backwards compatible, but may expose bugs in dak, debian-cd, apt and
> other tools, so IIRC this has been vetoed by the archive and CD teams.
> This also wouldn't result in users without non-free getting any of the
> non-free firmware, which maybe we want since it is the new default?

libapt or probably most user-facing client are no concern in this as
packages coming from multiple sources is normal. If you have e.g.
unstable and testing in your sources.list it is happening all the time,
or you have multiple mirrors, … only by the virtue of having packages
installed and somewhere online a client needs a way to figure out if the
version installed is the same as (one of) the versions online and which
online sources talk about the same version (naively, you would think
version number is enough, but libapt actually goes beyond that).

I assume through that in servers and other tools who are not usually
exposed to packages in multiple versions or multiple sources in general
have a harder time with this, so I can understand them being reluctant.

There are also a lot of tools… most user-facing clients will be based
on libapt or at least directly inspired by it, but if clients like
(c)debootstrap expect a package name to be no longer unique in their
world view (after all, they don't even do multi-sources like -updates
and -security …) I honestly don't know and fear the worst.


It is also that if we decide that, we basically decide that it will stay
that way forever. Its also an (arguably tiny) waste of diskspace and
such for everyone who has both components configured.

It is the only solution treating everyone equal though. All the others
talk only about sources.list with no idea if and how e.g. debootstrap
needs to understand that non-free-firmware is a thing now. Does it?


> 3. Add some code to apt to download non-free-firmware when non-free is
> available in the sources and the downloaded Release files. This would
> solve the issue for Debian and all other derivatives too, if they
> decide to add a new non-free-firmware component too. This might not
> be accepted by apt developers as it is kind of a hack to special-case
> Debian component semantics in apt, although maybe a component mapping
> config option would be accepted. This might result in extra Debian
> support requests when users notice the new component in `apt update`.
> This might not work for users of tools not based on apt, like cupt?
> This wouldn't result in users without non-free getting any non-free
> firmware though, which maybe we want since it is the new default?

The problem with this within libapt is that libapt wants to look at the
sources.list entry and produce a list of files which could possibly
belong to this entry. The Release file is just one of those files.
It is consulted later to remove entries from the file list (e.g. in an
'update'), but files aren't added at that stage.

Think of 'apt update --print-uris': What would that print if it would
need to look at the Release first? If you look closely, you will notice
e.g. a line talking about 'binary-all/Packages'. The Release file will
later tell apt to not download it for Debian. Similar, depending on your
locale environment apt talks about downloading Translation files here
which don't exist or perhaps even of Contents files which are in an
other location than in reality for some distros (hello Ubuntu).

So, if we wanted that, I could hardcode in apt that it should look for
non-free-firmware, too, if it sees non-free as a component configured
and decide later on not to download that component if it doesn't exist –
on the upside (depending on your view) that isn't Debian specific.
In practice it would probably turn out to be a medium sized patch as
so far apt doesn't have the concept of an implicit component, but it
shouldn't be rocket science and easily done ahead of the freeze.
It would probably be too big for a backport through and even if very
unlikely in practice a behaviour change as suddenly grabbing an existing
non-free-firmware component and upgrades from it after an unattended
upgrade in an unattended upgrade is… that rules out availability of
the bookworm-version of firmware packages in the upgrade to bookworm.
They would be installed only with the first 'update && upgrade' cycle
on the upgraded bookworm system, potentially unattended. Most other
"solutions" have the same "problem" – I would hope that for firmware
packages it isn't really one in practice as they should be very light on
(rev-)dependencies and demands upon them.

We are potentially burning one compon

Re: Firmware GR result - what happens next?

[Pascal Hambourg]

On 13/10/2022 at 10:13, Cyril Brulebois wrote:

Pascal Hambourg  (2022-10-13):

What about firmware not available in Debian packages ? For example
Prims54 wireless adapters using p54pci and p54usb drivers included
in nic-wireless-modules-*-di need firmware unavailable in Debian
packages.


As far as I understand it, they can be packaged once their
redistributability is clarified?


What if some firmware is not redistributable in the end ?

The above example is for old hardware and the download URLs are 
mentioned in the Debian wiki, so I assumed that the reason why it was 
not packaged was because it was not redistributable. But maybe I was 
wrong and nobody just cared.


Also, I assume that packages such as firmware-b43legacy-installer and 
firmware-b43-installer which download Broadcom drivers and extract the 
firmware from them would not need to exist if the firmware was known to 
be redistributable. But I may be wrong again.

Re: Firmware GR result - what happens next?

[Holger Wansing]

Hi,

Pascal Hambourg  wrote (Fri, 14 Oct 2022 19:50:11 
+0200):
> On 13/10/2022 at 10:13, Cyril Brulebois wrote:
> > Pascal Hambourg  (2022-10-13):
> >> What about firmware not available in Debian packages ? For example
> >> Prims54 wireless adapters using p54pci and p54usb drivers included
> >> in nic-wireless-modules-*-di need firmware unavailable in Debian
> >> packages.
> > 
> > As far as I understand it, they can be packaged once their
> > redistributability is clarified?

The only way for the installer to make use of such firmware is, if the 
user provides the firmware files on a removable device, like an USB stick.
This is documented here:
https://d-i.debian.org/manual/en.amd64/ch06s04.html

How and where to get the files, is not where Debian can help with in this case.


Holger


-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Re: Firmware GR result - what happens next?

[Pascal Hambourg]

On 14/10/2022 at 20:54, Holger Wansing wrote:


Pascal Hambourg  wrote (Fri, 14 Oct 2022 19:50:11 
+0200):

On 13/10/2022 at 10:13, Cyril Brulebois wrote:

Pascal Hambourg  (2022-10-13):

What about firmware not available in Debian packages ? For example
Prims54 wireless adapters using p54pci and p54usb drivers included
in nic-wireless-modules-*-di need firmware unavailable in Debian
packages.


As far as I understand it, they can be packaged once their
redistributability is clarified?


The only way for the installer to make use of such firmware is, if the
user provides the firmware files on a removable device, like an USB stick.


This is my point. If Debian does not provide all firmware for drivers 
which are included into the installer, then support for extra firmware 
medium is still useful and should not be removed.

Re: Firmware GR result - what happens next?

[Holger Wansing]

Hi,

Pascal Hambourg  wrote (Fri, 14 Oct 2022 21:31:31 
+0200):
> >>> As far as I understand it, they can be packaged once their
> >>> redistributability is clarified?
> > 
> > The only way for the installer to make use of such firmware is, if the
> > user provides the firmware files on a removable device, like an USB stick.
> 
> This is my point. If Debian does not provide all firmware for drivers 
> which are included into the installer, then support for extra firmware 
> medium is still useful and should not be removed.

For this specific case (use of firmware files, which are not available in 
Debian) there is no need for a special installer medium.
The usual installer also has the capability to make use of firmware
provided via USB.


Holger



-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Re: Firmware GR result - what happens next?

[Cyril Brulebois]

Holger Wansing  (2022-10-14):
> Pascal Hambourg  wrote (Fri, 14 Oct 2022
> > This is my point. If Debian does not provide all firmware for
> > drivers which are included into the installer, then support for
> > extra firmware medium is still useful and should not be removed.
> 
> For this specific case (use of firmware files, which are not available
> in Debian) there is no need for a special installer medium.  The usual
> installer also has the capability to make use of firmware provided via
> USB.

That's really Pascal's point: we talked about maybe removing that.

(It's hard to get right for end-users.)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Firmware GR result - what happens next?

[Pascal Hambourg]

On 14/10/2022 at 21:49, Holger Wansing wrote:


Pascal Hambourg  wrote (Fri, 14 Oct 2022 21:31:31 
+0200):

As far as I understand it, they can be packaged once their
redistributability is clarified?


The only way for the installer to make use of such firmware is, if the
user provides the firmware files on a removable device, like an USB stick.


This is my point. If Debian does not provide all firmware for drivers
which are included into the installer, then support for extra firmware
medium is still useful and should not be removed.


For this specific case (use of firmware files, which are not available in
Debian) there is no need for a special installer medium.


I did not mention the need for any special installer medium, only the 
need for the installer to support extra firmware medium.



The usual installer also has the capability to make use of firmware
provided via USB.


It was suggested in this thread to remove this feature.

Re: Firmware GR result - what happens next?

[Holger Wansing]

Hi,

Pascal Hambourg  wrote (Fri, 14 Oct 2022 21:57:37 
+0200):
> I did not mention the need for any special installer medium, only the 
> need for the installer to support extra firmware medium.
> 
> > The usual installer also has the capability to make use of firmware
> > provided via USB.
> 
> It was suggested in this thread to remove this feature.

Hmm, ok. 
You talked about an "extra firmware medium".
I mixed that up with the "extra installer images with firmware included".

Sorry for the noise

Holger


-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076