Verification logs success noble.
csmith@midtown:~$ ./sru-2075337.sh
+ cat
+ SET_LOCALE_YAML='#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
locale: fr_FR
'
+ export LANG=en_AU.ISO-8859-1
+ LANG=en_AU.ISO-8859-1
+ for RELEASE in noble
+ NAME=sru-2075337-noble
+ lxc launch ubuntu:noble sru-2075337-noble -c
'cloud-init.user-data=#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
locale: fr_FR
'
Creating sru-2075337-noble
Starting sru-2075337-noble
+ sleep 10
+ lxc exec sru-2075337-noble -- dpkg-query -W python3-minimal
python3-minimal 3.12.3-0ubuntu1
+ echo 'Run: sudo apt install --reinstall cloud-init to validate failure'
Run: sudo apt install --reinstall cloud-init to validate failure
+ lxc console sru-2075337-noble
To detach from the console, press: <ctrl>+a q
sru-2075337-noble login: ubuntu
Password:
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.5.0-44-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of jeu. 22 ao�t 2024 21:36:55 UTC
System load: 2.21
Usage of /: 53.9% of 839MB
Memory usage: 0%
Swap usage: 0%
Temperature: 52.0 C
Processes: 20
Users logged in: 0
IPv4 address for eth0: 10.125.221.103
IPv6 address for eth0: fd42:e810:4b9b:718:216:3eff:fedf:1d30
Expanded Security Maintenance for Applications is not enabled.
0 updates can be applied immediately.
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@sru-2075337-noble:~$ sudo apt-get install --reinstall cloud-init
Lecture des listes de paquets... Fait
Construction de l'arbre des d�pendances... Fait
Lecture des informations d'�tat... Fait
0 mis � jour, 0 nouvellement install�s, 1 r�install�s, 0 � enlever et 0 non mis
� jour.
Il est n�cessaire de prendre 598 ko dans les archives.
Apr�s cette op�ration, 0 o d'espace disque suppl�mentaires seront utilis�s.
R�ception de�:1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64
cloud-init all 24.1.3-0ubuntu3.3 [598 kB]
598 ko r�ceptionn�s en 0s (1�398 ko/s)
Pr�configuration des paquets...
(Lecture de la base de donn�es... 34407 fichiers et r�pertoires d�j� install�s.)
Pr�paration du d�paquetage de .../cloud-init_24.1.3-0ubuntu3.3_all.deb ...
Traceback (most recent call last):
File "/usr/bin/py3clean", line 210, in <module>
main()
File "/usr/bin/py3clean", line 196, in main
pfiles = set(dpf.from_package(options.package))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: avertissement: le sous-processus ancien paquet cloud-init script
pre-removal a renvoy� un �tat de sortie d'erreur 1
dpkg: tentative d'ex�cution du script du nouveau paquet � la place...
Traceback (most recent call last):
File "/usr/bin/py3clean", line 210, in <module>
main()
File "/usr/bin/py3clean", line 196, in main
pfiles = set(dpf.from_package(options.package))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: erreur de traitement de l'archive
/var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb (--unpack)�:
le sous-processus nouveau cloud-init paquet pre-removal script a renvoy� un
�tat de sortie d'erreur 1
dmesg: read kernel buffer failed: Operation not permitted
Traceback (most recent call last):
File "/usr/bin/py3compile", line 323, in <module>
main()
File "/usr/bin/py3compile", line 302, in main
compile(files, versions,
File "/usr/bin/py3compile", line 185, in compile
for fn, versions_to_compile in filter_files(files, e_patterns, versions):
File "/usr/bin/py3compile", line 128, in filter_files
for fpath in files:
File "/usr/share/python3/debpython/files.py", line 71, in filter_public
for fn in files:
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: erreur lors du nettoyage:
le sous-processus paquet cloud-init script post-installation install� a
renvoy� un �tat de sortie d'erreur 1
Des erreurs ont �t� rencontr�es pendant l'ex�cution�:
/var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
ubuntu@sru-2075337-noble:~$ echo $?
100
ubuntu@sru-2075337-noble:~$
logout
Ubuntu 24.04 LTS sru-2075337-noble console
sru-2075337-noble login:
+ lxc delete sru-2075337-noble --force
+ echo 'Validate success when using python3-minimal from -proposed'
Validate success when using python3-minimal from -proposed
+ lxc launch ubuntu:noble sru-2075337-noble -c
'cloud-init.user-data=#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
locale: fr_FR
'
Creating sru-2075337-noble
Starting sru-2075337-noble
+ lxc file push setup_proposed.sh sru-2075337-noble/
+ lxc exec sru-2075337-noble -- bash /setup_proposed.sh
+ lxc exec sru-2075337-noble -- apt-get install python3-minimal -y -t
noble-proposed
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libpython3-stdlib python3
Suggested packages:
python3-doc python3-tk python3-venv
The following packages will be upgraded:
libpython3-stdlib python3 python3-minimal
3 upgraded, 0 newly installed, 0 to remove and 105 not upgraded.
Need to get 60.4 kB of archives.
After this operation, 1024 B disk space will be freed.
Get:1 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64
python3-minimal amd64 3.12.3-0ubuntu2 [27.4 kB]
Get:2 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64 python3 amd64
3.12.3-0ubuntu2 [23.0 kB]
Get:3 http://archive.ubuntu.com/ubuntu noble-proposed/main amd64
libpython3-stdlib amd64 3.12.3-0ubuntu2 [10.0 kB]
Fetched 60.4 kB in 1s (66.7 kB/s)
(Reading database ... 34407 files and directories currently installed.)
Preparing to unpack .../python3-minimal_3.12.3-0ubuntu2_amd64.deb ...
Unpacking python3-minimal (3.12.3-0ubuntu2) over (3.12.3-0ubuntu1) ...
Setting up python3-minimal (3.12.3-0ubuntu2) ...
(Reading database ... 34407 files and directories currently installed.)
Preparing to unpack .../python3_3.12.3-0ubuntu2_amd64.deb ...
running python pre-rtupdate hooks for python3.12...
Unpacking python3 (3.12.3-0ubuntu2) over (3.12.3-0ubuntu1) ...
Preparing to unpack .../libpython3-stdlib_3.12.3-0ubuntu2_amd64.deb ...
Unpacking libpython3-stdlib:amd64 (3.12.3-0ubuntu2) over (3.12.3-0ubuntu1) ...
Setting up libpython3-stdlib:amd64 (3.12.3-0ubuntu2) ...
Setting up python3 (3.12.3-0ubuntu2) ...
running python rtupdate hooks for python3.12...
running python post-rtupdate hooks for python3.12...
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
+ lxc exec sru-2075337-noble -- dpkg-query -W python3-minimal
python3-minimal 3.12.3-0ubuntu2
+ lxc console sru-2075337-noble
To detach from the console, press: <ctrl>+a q
sru-2075337-noble login: ubuntu
Password:
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.5.0-44-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of jeu. 22 ao�t 2024 21:37:34 UTC
System load: 2.87
Usage of /: 67.8% of 829MB
Memory usage: 0%
Swap usage: 0%
Temperature: 57.0 C
Processes: 21
Users logged in: 0
IPv4 address for eth0: 10.125.221.18
IPv6 address for eth0: fd42:e810:4b9b:718:216:3eff:fe66:1668
Expanded Security Maintenance for Applications is not enabled.
4 updates can be applied immediately.
3 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@sru-2075337-noble:~$ sudo apt-get install --reinstall cloud-init
Lecture des listes de paquets... Fait
Construction de l'arbre des d�pendances... Fait
Lecture des informations d'�tat... Fait
0 mis � jour, 0 nouvellement install�s, 1 r�install�s, 0 � enlever et 61 non
mis � jour.
Il est n�cessaire de prendre 598 ko dans les archives.
Apr�s cette op�ration, 0 o d'espace disque suppl�mentaires seront utilis�s.
R�ception de�:1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64
cloud-init all 24.1.3-0ubuntu3.3 [598 kB]
598 ko r�ceptionn�s en 1s (520 ko/s)
Pr�configuration des paquets...
(Lecture de la base de donn�es... 34407 fichiers et r�pertoires d�j� install�s.)
Pr�paration du d�paquetage de .../cloud-init_24.1.3-0ubuntu3.3_all.deb ...
D�paquetage de cloud-init (24.1.3-0ubuntu3.3) sur (24.1.3-0ubuntu3.3) ...
echParam�trage de cloud-init (24.1.3-0ubuntu3.3) ...
o $Traitement des actions diff�r�es (��triggers��) pour man-db
(2.12.0-4build2)�...
?Traitement des actions diff�r�es (��triggers��) pour rsyslog
(8.2312.0-3ubuntu9)�...
Scanning processes...
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
ubuntu@sru-2075337-noble:~$ echo $?
0
ubuntu@sru-2075337-noble:~$
logout
Ubuntu 24.04 LTS sru-2075337-noble console
sru-2075337-noble login:
** Tags removed: verification-needed-noble
** Tags added: verification-done-noble
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to dh-python in Ubuntu.
https://bugs.launchpad.net/bugs/2075337
Title:
[SRU] py3clean fails when using alternate character set
Status in cloud-init package in Ubuntu:
New
Status in dh-python package in Ubuntu:
Invalid
Status in python3-defaults package in Ubuntu:
Fix Released
Status in ubuntu-advantage-tools package in Ubuntu:
New
Status in cloud-init source package in Jammy:
Invalid
Status in dh-python source package in Jammy:
Invalid
Status in python3-defaults source package in Jammy:
Fix Committed
Status in ubuntu-advantage-tools source package in Jammy:
Invalid
Status in cloud-init source package in Noble:
Fix Committed
Status in dh-python source package in Noble:
Invalid
Status in python3-defaults source package in Noble:
Fix Committed
Status in ubuntu-advantage-tools source package in Noble:
Fix Committed
Bug description:
[ Impact ]
With the current version of python3-defaults in jammy and noble,
packages which utilize py3compile and py3clean in their maintscripts
(which is more or less any package which includes Python modules), and
which also are affected by diversions installed by dpkg-divert, will
fail to install/upgrade/remove on systems with non-UTF-8 locales,
which include non-ASCII characters in their localized dpkg -L messages
(and which therefore produce output that is not valid UTF-8).
As this has the potential to break apt upgrade, the impact is quite
severe (especially if this occurs during a dist-upgrade). However, the
scope is probably *fairly* limited given this only affects systems
with non-UTF-8 locales (something that has not been the default for
many years), and which produce non-ASCII output (e.g. French
characters with diacriticals) in their localized messages for dpkg -L.
[ Rectification Plan ]
Firstly, python3-defaults must be patched in oracular, noble, and
jammy. doko has uploaded the fix upstream and synced this to oracular.
Benjamin has sponsored the noble and jammy SRUs.
Next, affected packages must be patched to include a "Breaks" line on
earlier versions of python3-minimal (the python3-defaults binary
package containing py3clean and py3compile).
Packages which install Python modules are easily selected, but those
affected by diversions are (computationally) impossible to select (see
comment 18 for details). Thus, determining the set of affected
packages was non-trivial, but Benjamin has kindly produced such a list
by empirical means (see comment 24). These will be added to the ticket
as they are uploaded.
[ Known affected packages ]
```
$ for section in main restricted universe multiverse; do echo "noble
$section:"; sqlite3 affected.sqlite3 "SELECT package, source FROM
package_affected WHERE section = '$section' and install_affected = 1 AND
(remove_affected = 1 OR remove_affected ISNULL) ORDER BY package;"; echo; done
noble main:
cloud-init|cloud-init
nvme-stas|nvme-stas
python3-rtslib-fb|python-rtslib-fb
targetcli-fb|targetcli-fb
ubiquity|ubiquity
ubiquity-frontend-debconf|ubiquity
ubiquity-frontend-gtk|ubiquity
ubuntu-drivers-common|ubuntu-drivers-common
ubuntu-pro-client|ubuntu-advantage-tools
walinuxagent|walinuxagent
noble restricted:
noble universe:
a2d|a2d
apertium-apy|apertium-apy
blueman|blueman
borgmatic|borgmatic
condor|condor
cpupower-gui|cpupower-gui
debomatic|debomatic
dell-recovery|dell-recovery
deluge-web|deluge
depthcharge-tools|depthcharge-tools
dhcpy6d|dhcpy6d
dkimpy-milter|dkimpy-milter
fenrir|fenrir
galileo|galileo
glances|glances
graphite-carbon|graphite-carbon
grokmirror|grokmirror
guerillabackup|guerillabackup
ionit|ionit
jupyterhub|jupyterhub
liquidctl|liquidctl
localslackirc|localslackirc
logdata-anomaly-miner|logdata-anomaly-miner
lomiri|lomiri
mailman3|mailman3
mailman3-full|mailman3
matrix-sydent|matrix-sydent
mom|mom
mopidy|mopidy
mpd-sima|mpd-sima
nordugrid-arc-arex|nordugrid-arc
notus-scanner|notus-scanner
nrpe-ng|nrpe-ng
onionprobe|onionprobe
openscap-daemon|openscap-daemon
ospd-openvas|ospd-openvas
pagekite|pagekite
pagure|pagure
pagure-ev-server|pagure
pagure-milters|pagure
pglistener|pglistener
policyd-rate-limit|policyd-rate-limit
postfix-mta-sts-resolver|postfix-mta-sts-resolver
powa-collector|powa-collector
prelude-correlator|prelude-correlator
prometheus-pgbouncer-exporter|prometheus-pgbouncer-exporter
prometheus-xmpp-alerts|prometheus-xmpp-alerts
pyroman|pyroman
python3-autopilot|autopilot
python3-btchip|btchip-python
python3-pyro5|pyro5
qweborf|weborf
razercfg|razercfg
sgt-launcher|sgt-launcher
sms4you|sms4you
supervisor|supervisor
syncplay-server|syncplay
timekpr-next|timekpr-next
twms|twms
ubiquity-frontend-kde|ubiquity
vanguards|vanguards
vaultlocker|vaultlocker
voctomix-outcasts|voctomix-outcasts
xdiagnose|xdiagnose
noble multiverse:
r4d|r4d
virtualbox|virtualbox
```
On Ubuntu 22.04 (jammy) only packages in universe are affected:
```
$ sqlite3 affected-jammy.sqlite3 "SELECT package, source FROM
package_affected WHERE install_affected = 1 AND (remove_affected = 1 OR
remove_affected ISNULL) ORDER BY package;"
ganeti-3.0|ganeti
sgt-launcher|sgt-launcher
```
Note: Due to time constraints, 3861/59351 (6 %) universe package
haven't been analyzed.
Code and resulting database can be found on
https://github.com/bdrung/bdrung-scripts/tree/lp-2075337-find-affected
[ Test Plan ]
Where $series is jammy or noble as appropriate, and $package is the
name of the affected package being SRU'd:
$ lxc launch ubuntu:$series py3clean-test
$ lxc shell py3clean-test
# locale-gen fr_FR
Generating locales (this might take a while)...
fr_FR.ISO-8859-1... done
Generation complete.
# update-locale LANG=fr_FR
# exit
# adjust terminal to ISO-8859-1
$ lxc shell py3clean-test
# apt update && apt install --reinstall $package
With the current version of python3-defaults in jammy and noble this
should fail at the final apt install --reinstall step. With the
proposed version, it should succeed.
[ What Could Go Wrong? ]
The potential for regression is low since the python3-defaults changes
are minimal and have been reasonably tested, and the only change to
affected packages is the addition of a Breaks clause (i.e. the content
of the packages are unaffected, just the meta-data).
The major cause for concern is that we are not 100% certain that the
Breaks addition is sufficient to fix this. The Debian policy manual is
ambiguous as to whether Breaks implies that the fixed package must be
unpacked *before* the prerm maintscript of the package it breaks is
executed; the policy manual mentions this must happen before the
"unpack" phase, but prerm scripts run *before* the unpack phase.
However, in all experiments run so far, this did indeed resolve the
problem.
Still, as an extra hedge against failure here, we intend that the
python3-defaults SRU is released ASAP, and that we then delay a week
to allow it to propagate before releasing SRUs for the affected
packages.
[ Original Description ]
We noticed this in bug 2060769 but think it is a separate issue.
dh_python3 is putting a py3clean invocation into package prerm
maintainer scripts. When that runs, it explodes if the system
character set is for example ISO-8859-1.
This appears to affect every package for which dh_python3 has inserted
a call to py3clean into the package prerm.
python3-minimal 3.12.3-0ubuntu1
In this reproducer, cloud-init is at 24.1.3-0ubuntu3.3.
Steps to reproduce:
$ lxc launch ubuntu:noble rbasak-pro-l8n
$ lxc shell rbasak-pro-l8n
# locale-gen fr_FR
Generating locales (this might take a while)...
fr_FR.ISO-8859-1... done
Generation complete.
# update-locale LANG=fr_FR
# exit
# adjust terminal to ISO-8859-1
$ lxc shell rbasak-pro-l8n
# apt update && apt install --reinstall cloud-init
Expected results: success
Actual results:
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances... Fait
Lecture des informations d'état... Fait
0 mis à jour, 0 nouvellement installés, 1 réinstallés, 0 à enlever et 5 non
mis à jour.
Il est nécessaire de prendre 598 ko dans les archives.
Après cette opération, 0 o d'espace disque supplémentaires seront utilisés.
Réception de :1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64
cloud-init all 24.1.3-0ubuntu3.3 [598 kB]
598 ko réceptionnés en 1s (926 ko/s)
Préconfiguration des paquets...
(Lecture de la base de données... 34406 fichiers et répertoires déjà
installés.)
Préparation du dépaquetage de .../cloud-init_24.1.3-0ubuntu3.3_all.deb ...
Traceback (most recent call last):
File "/usr/bin/py3clean", line 210, in <module>
main()
File "/usr/bin/py3clean", line 196, in main
pfiles = set(dpf.from_package(options.package))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: avertissement: le sous-processus ancien paquet cloud-init script
pre-removal a renvoyé un état de sortie d'erreur 1
dpkg: tentative d'exécution du script du nouveau paquet à la place...
Traceback (most recent call last):
File "/usr/bin/py3clean", line 210, in <module>
main()
File "/usr/bin/py3clean", line 196, in main
pfiles = set(dpf.from_package(options.package))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: erreur de traitement de l'archive
/var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb (--unpack) :
le sous-processus nouveau cloud-init paquet pre-removal script a renvoyé un
état de sortie d'erreur 1
dmesg: read kernel buffer failed: Operation not permitted
Traceback (most
recent call last):
File "/usr/bin/py3compile", line 323, in <module>
main()
File "/usr/bin/py3compile", line 302, in main
compile(files, versions,
File "/usr/bin/py3compile", line 185, in compile
for fn, versions_to_compile in filter_files(files, e_patterns, versions):
File "/usr/bin/py3compile", line 128, in filter_files
for fpath in files:
File "/usr/share/python3/debpython/files.py", line 71, in filter_public
for fn in files:
File "/usr/share/python3/debpython/files.py", line 54, in from_package
stdout = str(stdout, 'utf-8')
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417:
invalid continuation byte
dpkg: erreur lors du nettoyage:
le sous-processus paquet cloud-init script post-installation installé a
renvoyé un état de sortie d'erreur 1
Des erreurs ont été rencontrées pendant l'exécution :
/var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2075337/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
