** Description changed:
[ Impact ]
If for any reason the user's system has any non-utf8 character as part
of the `apt list --installed` command, the Pro Client will fail to parse
the return output and break.
Independent of why this happens, the Pro Client should not be relying on
CLI output when there are Python bindings available to perform the same
task - and python3-apt provides ways to list installed packages.
The fix to this bug is to rely on the programmatic integration with APT
instead of calling the CLI.
-
[ Test Plan ]
Listing installed packages is part of the regular operation of the Pro
Client in many scenarios covered by the integration / acceptance tests.
For this particular issue:
- Launch a container for the release to be tested
- - make `apt list --installed` return a non-utf8 character (can be done by
mocking the `apt` binary in the system)
+ - Install a non-utf-8 locale and set it as the default. e.g.
+ - sudo locale-gen fr_FR
+ - sudo dpkg-reconfigure locales # select fr_FR
+ - # log out and log in
+ - pro status
- see it fail
- install the package from -proposed
- see it does not fail anymore
-
[ Where problems could occur ]
Switching from the CLI call to python3-apt brings the risk of wrong
implementation of the integration, which would lead to a different
package list to be returned by python3-apt. The regression potential is
mitigated by extensive testing - unit, integration and acceptance tests
rely on this functionality.
-
[ Original description ]
["2024-04-10T08:28:17.952", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:28:17.953", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:28:18.575", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 1475, in
action_status\n status, ret = actions.status(\n File
\"/usr/lib/python3/dist-package
s/uaclient/actions.py\", line 173, in status\n status =
ua_status.status(cfg=cfg, show_all=show_all)\n File
\"/usr/lib/python3/dist-packages/uaclient/
status.py\", line 418, in status\n response = _unattached_status(cfg)\n
File \"/usr/lib/python3/dist-packages/uaclient/status.py\", line 283, in _unat
tached_status\n resources = get_available_resources(cfg)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 680, in
get_available_re
sources\n resources = client.available_resources()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 114, in
available_resources\n
activity_info = self._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"deskt
op\": system.is_desktop(),\n File
\"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in
is_desktop\n for package in apt.get_installed_pa
ckages():\n File \"/usr/lib/python3/dist-packages/uaclient/apt.py\", line
727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"
--installed\"])\n File
\"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-pa
ckages/uaclient/system.py\", line 649, in _subp\n out_result =
out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't
decode b
yte 0xe1 in position 109: invalid continuation byte"}]
["2024-04-10T08:30:25.404", "DEBUG", "ubuntupro.config", "parse_config", 625,
"Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:30:25.408", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:30:25.409", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:30:25.421", "DEBUG", "ubuntupro.http", "configure_web_proxy",
146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.http", "configure_web_proxy",
154, "Setting global proxy dict", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed
with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin
lock starting for pro attach", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmparyyoirn", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmpn1yxl91o", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpwqyq85v1", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/machine-id", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:30:25.425", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file:
/run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:30:25.971", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n
return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n
File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in
action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File
\"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n
File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in
add_contract_machine\n activity_info = se
lf._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line
397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in
get_installed_packages\n out, _ = system.subp([\"apt\", \"list\",
\"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out
else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
["2024-04-10T08:38:29.763", "DEBUG", "ubuntupro.config", "parse_config", 625,
"Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:38:29.767", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:38:29.768", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy",
146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy",
154, "Setting global proxy dict", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed
with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin
lock starting for pro attach", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmplx6ehdag", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmp6ml_ltfj", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpvxf_b1v0", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/machine-id", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:38:29.784", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:38:30.366", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file:
/run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:38:30.367", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:38:30.367", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n
return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n
File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in
action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File
\"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n
File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in
add_contract_machine\n activity_info = se
lf._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line
397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in
get_installed_packages\n out, _ = system.subp([\"apt\", \"list\",
\"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out
else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubuntu-advantage-tools 31.2~20.04
ProcVersionSignature: Ubuntu 5.15.0-101.111~20.04.1-generic 5.15.143
Uname: Linux 5.15.0-101-generic x86_64
NonfreeKernelModules: falcon_lsm_serviceable falcon_nf_netcontain falcon_kal
falcon_lsm_pinned_16407 falcon_lsm_pinned_16206
ApportVersion: 2.20.11-0ubuntu27.27
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed Apr 10 08:28:11 2024
PackageArchitecture: all
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=es_ES
SHELL=/bin/bash
SourcePackage: ubuntu-advantage-tools
UpgradeStatus: No upgrade log present (probably fresh install)
apparmor_logs.txt:
cloud-id.txt-error: Invalid command specified 'cloud-id'.
livepatch-status.txt-error: Invalid command specified
'/snap/bin/canonical-livepatch status'.
ua-status.json-error: Failed running command 'pro status --format json'
[exit(1)]. Message:
uaclient.conf:
contract_url: https://contracts.canonical.com
log_level: debug
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2060769
Title:
pro-client execs `apt list --installed` instead of using python3-apt -
leading to utf-8 errors
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2060769/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
