Marking dh_python as invalid; it was added originally on the theory that we may have needed to patch the prerm fallback but this has thankfully turned out to be unnecessary.
** Description changed: + [ Impact ] + + With the current version of python3-defaults in jammy and noble, + packages which utilize py3compile and py3clean in their maintscripts + (which is more or less any package which includes Python modules), and + which also are affected by diversions installed by dpkg-divert, will + fail to install/upgrade/remove on systems with non-UTF-8 locales, which + include non-ASCII characters in their localized dpkg -L messages (and + which therefore produce output that is not valid UTF-8). + + As this has the potential to break apt upgrade, the impact is quite + severe (especially if this occurs during a dist-upgrade). However, the + scope is probably *fairly* limited given this only affects systems with + non-UTF-8 locales (something that has not been the default for many + years), and which produce non-ASCII output (e.g. French characters with + diacriticals) in their localized messages for dpkg -L. + + [ Rectification Plan ] + + Firstly, python3-defaults must be patched in oracular, noble, and jammy. + doko has uploaded the fix upstream and synced this to oracular. Benjamin + has sponsored the noble and jammy SRUs. + + Next, affected packages must be patched to include a "Breaks" line on + earlier versions of python3-minimal (the python3-defaults binary package + containing py3clean and py3compile). + + Packages which install Python modules are easily selected, but those + affected by diversions are (computationally) impossible to select (see + comment 18 for details). Thus, determining the set of affected packages + was non-trivial, but Benjamin has kindly produced such a list by + empirical means (see comment 24). These will be added to the ticket as + they are uploaded. + + [ Test Plan ] + + Where $series is jammy or noble as appropriate, and $package is the name + of the affected package being SRU'd: + + $ lxc launch ubuntu:$series py3clean-test + $ lxc shell py3clean-test + # locale-gen fr_FR + Generating locales (this might take a while)... + fr_FR.ISO-8859-1... done + Generation complete. + # update-locale LANG=fr_FR + # exit + # adjust terminal to ISO-8859-1 + $ lxc shell py3clean-test + # apt update && apt install --reinstall $package + + With the current version of python3-defaults in jammy and noble this + should fail at the final apt install --reinstall step. With the proposed + version, it should succeed. + + [ What Could Go Wrong? ] + + The potential for regression is low since the python3-defaults changes + are minimal and have been reasonably tested, and the only change to + affected packages is the addition of a Breaks clause (i.e. the content + of the packages are unaffected, just the meta-data). + + The major cause for concern is that we are not 100% certain that the + Breaks addition is sufficient to fix this. The Debian policy manual is + ambiguous as to whether Breaks implies that the fixed package must be + unpacked *before* the prerm maintscript of the package it breaks is + executed; the policy manual mentions this must happen before the + "unpack" phase, but prerm scripts run *before* the unpack phase. + However, in all experiments run so far, this did indeed resolve the + problem. + + Still, as an extra hedge against failure here, we intend that the + python3-defaults SRU is released ASAP, and that we then delay a week to + allow it to propagate before releasing SRUs for the affected packages. + + [ Original Description ] + We noticed this in bug 2060769 but think it is a separate issue. dh_python3 is putting a py3clean invocation into package prerm maintainer scripts. When that runs, it explodes if the system character set is for example ISO-8859-1. This appears to affect every package for which dh_python3 has inserted a call to py3clean into the package prerm. python3-minimal 3.12.3-0ubuntu1 In this reproducer, cloud-init is at 24.1.3-0ubuntu3.3. Steps to reproduce: $ lxc launch ubuntu:noble rbasak-pro-l8n $ lxc shell rbasak-pro-l8n # locale-gen fr_FR Generating locales (this might take a while)... - fr_FR.ISO-8859-1... done + fr_FR.ISO-8859-1... done Generation complete. # update-locale LANG=fr_FR # exit # adjust terminal to ISO-8859-1 $ lxc shell rbasak-pro-l8n # apt update && apt install --reinstall cloud-init Expected results: success Actual results: Lecture des listes de paquets... Fait Construction de l'arbre des dépendances... Fait - Lecture des informations d'état... Fait + Lecture des informations d'état... Fait 0 mis à jour, 0 nouvellement installés, 1 réinstallés, 0 à enlever et 5 non mis à jour. Il est nécessaire de prendre 598 ko dans les archives. Après cette opération, 0 o d'espace disque supplémentaires seront utilisés. Réception de :1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 cloud-init all 24.1.3-0ubuntu3.3 [598 kB] 598 ko réceptionnés en 1s (926 ko/s) Préconfiguration des paquets... (Lecture de la base de données... 34406 fichiers et répertoires déjà installés.) Préparation du dépaquetage de .../cloud-init_24.1.3-0ubuntu3.3_all.deb ... Traceback (most recent call last): - File "/usr/bin/py3clean", line 210, in <module> - main() - File "/usr/bin/py3clean", line 196, in main - pfiles = set(dpf.from_package(options.package)) - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "/usr/share/python3/debpython/files.py", line 54, in from_package - stdout = str(stdout, 'utf-8') - ^^^^^^^^^^^^^^^^^^^^ + File "/usr/bin/py3clean", line 210, in <module> + main() + File "/usr/bin/py3clean", line 196, in main + pfiles = set(dpf.from_package(options.package)) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/usr/share/python3/debpython/files.py", line 54, in from_package + stdout = str(stdout, 'utf-8') + ^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte dpkg: avertissement: le sous-processus ancien paquet cloud-init script pre-removal a renvoyé un état de sortie d'erreur 1 dpkg: tentative d'exécution du script du nouveau paquet à la place... Traceback (most recent call last): - File "/usr/bin/py3clean", line 210, in <module> - main() - File "/usr/bin/py3clean", line 196, in main - pfiles = set(dpf.from_package(options.package)) - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - File "/usr/share/python3/debpython/files.py", line 54, in from_package - stdout = str(stdout, 'utf-8') - ^^^^^^^^^^^^^^^^^^^^ + File "/usr/bin/py3clean", line 210, in <module> + main() + File "/usr/bin/py3clean", line 196, in main + pfiles = set(dpf.from_package(options.package)) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/usr/share/python3/debpython/files.py", line 54, in from_package + stdout = str(stdout, 'utf-8') + ^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte dpkg: erreur de traitement de l'archive /var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb (--unpack) : - le sous-processus nouveau cloud-init paquet pre-removal script a renvoyé un état de sortie d'erreur 1 + le sous-processus nouveau cloud-init paquet pre-removal script a renvoyé un état de sortie d'erreur 1 dmesg: read kernel buffer failed: Operation not permitted - Traceback (most recent call last): - File "/usr/bin/py3compile", line 323, in <module> - main() - File "/usr/bin/py3compile", line 302, in main - compile(files, versions, - File "/usr/bin/py3compile", line 185, in compile - for fn, versions_to_compile in filter_files(files, e_patterns, versions): - File "/usr/bin/py3compile", line 128, in filter_files - for fpath in files: - File "/usr/share/python3/debpython/files.py", line 71, in filter_public - for fn in files: - File "/usr/share/python3/debpython/files.py", line 54, in from_package - stdout = str(stdout, 'utf-8') - ^^^^^^^^^^^^^^^^^^^^ + Traceback (most recent call last): + File "/usr/bin/py3compile", line 323, in <module> + main() + File "/usr/bin/py3compile", line 302, in main + compile(files, versions, + File "/usr/bin/py3compile", line 185, in compile + for fn, versions_to_compile in filter_files(files, e_patterns, versions): + File "/usr/bin/py3compile", line 128, in filter_files + for fpath in files: + File "/usr/share/python3/debpython/files.py", line 71, in filter_public + for fn in files: + File "/usr/share/python3/debpython/files.py", line 54, in from_package + stdout = str(stdout, 'utf-8') + ^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2417: invalid continuation byte dpkg: erreur lors du nettoyage: - le sous-processus paquet cloud-init script post-installation installé a renvoyé un état de sortie d'erreur 1 + le sous-processus paquet cloud-init script post-installation installé a renvoyé un état de sortie d'erreur 1 Des erreurs ont été rencontrées pendant l'exécution : - /var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb + /var/cache/apt/archives/cloud-init_24.1.3-0ubuntu3.3_all.deb E: Sub-process /usr/bin/dpkg returned an error code (1) ** Changed in: python3-defaults (Ubuntu) Status: New => Confirmed ** Changed in: python3-defaults (Ubuntu Jammy) Importance: Undecided => Critical ** Changed in: python3-defaults (Ubuntu Jammy) Status: New => Confirmed ** Changed in: python3-defaults (Ubuntu Noble) Importance: Undecided => Critical ** Changed in: python3-defaults (Ubuntu Noble) Status: New => Confirmed ** Changed in: dh-python (Ubuntu) Status: New => Invalid ** Changed in: dh-python (Ubuntu Jammy) Status: New => Invalid ** Changed in: dh-python (Ubuntu Noble) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2075337 Title: [SRU] py3clean fails when using alternate character set To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2075337/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
