** Description changed:
- Package reporter from landscape client is failing to report to server
- because seemingly of a lsb parse issue.
+ [ Impact ]
+
+ * Error when parsing lsb_release, landscape-package-reporter unable to
+ report package list to server (LP: #2031036):
+
+ * The version of Landscape Client in the Ubuntu main repository for
+ Focal and Jammy fails to report package information Landscape Server
+ when the client machine has lsb modules installed (ex. lsb-core).
+
+ [ Test Plan ]
+
+ * Perform the following for both Focal and Jammy.
+
+ * `lxc launch ubuntu:focal # or ubuntu:jammy`
+
+ * `lxc exec <lxc-container-name> -- su -l ubuntu # shell into
+ container`
+
+ * `sudo apt update && sudo apt install -y landscape-client`
+
+ * `lsb_release -a # Check no lsb modules`
+
+ * `sudo apt-get install -y lsb-core # install lsb module`
+
+ * `sudo landscape-config --silent --url="$LANDSCAPE_ROOT_URL/message-
+ system" --ping-url="$LANDSCAPE_ROOT_URL/ping" --account-name
+ $ACCOUNT_NAME --registration-key $REGISTRATION_KEY --computer-title
+ $HOSTNAME`
+
+ * Accept the machine's registration if needed and wait for package
+ reporting to (try and) run:
+
+ * tail /var/log/landscape/package-reporter.log
+
+ * You should see `ValueError: too many values to unpack`. Now remove
+ this container from Landscape and delete it.
+
+ * Create a fresh container of the series and add my ppa:
+
+ * sudo add-apt-repository ppa:jansdhillon/landscape-client-dev &&
+ sudo apt update
+
+ * Now, install an LSB module, `psutil`, and the proposed Landscape
+ Client:
+
+ * sudo apt install -y lsb-core python3-psutil landscape-client #
+ should pull new version
+
+ * Replace line 601 (602 on Jammy, 596 if not proposed version) of
+ `/usr/lib/python3/dist-packages/landscape/client/package/reporter.py`
+ with the following to apply the benchmarking to the function which
+ computes changes in packages to report back to Landscape Server:
+
+ def _compute_packages_changes(self): # noqa: max-complexity: 13
+ import cProfile
+ import pstats
+ from datetime import datetime
+ import psutil
+
+ profile = cProfile.Profile()
+ process = psutil.Process()
+ start_cpu_times = process.cpu_times()
+ profile.enable()
+
+ result = self.compute_packages_change_inner()
+
+ end_cpu_times = process.cpu_times()
+
+ profile.disable()
+
+ user_time = end_cpu_times.user - start_cpu_times.user
+ system_time = end_cpu_times.system - start_cpu_times.system
+ total_cpu_time = user_time + system_time
+
+ output_path = "/var/lib/landscape/client/result.txt"
+ with open(output_path, "a") as fp:
+ now = datetime.now()
+ fp.write(f"\n--------- Run on: {now.strftime('%Y-%m-%d
%H:%M:%S')} ---------\n\n")
+ stats = pstats.Stats(profile, stream=fp)
+ stats.strip_dirs().sort_stats("cumulative").print_stats(10)
+ fp.write(f"CPU Time: {total_cpu_time}s\n")
+ return result
+
+ def compute_packages_change_inner(self):
+
+ * Now register the container with Landscape Server, wait for package-
+ reporter to run again and verify no errors are raised:
+
+ * sudo landscape-config --silent
+ --url="$LANDSCAPE_ROOT_URL/message-system" --ping-
+ url="$LANDSCAPE_ROOT_URL/ping" --account-name $ACCOUNT_NAME
+ --registration-key $REGISTRATION_KEY --computer-title $HOSTNAME
+
+ * sudo watch tail -n 30 /var/log/landscape/package-reporter.log
+
+ * Wait for package reporter to run and check the benchmarking results
+ for the proposed version:
+
+ * less /var/lib/landscape/client/result.txt
+
+ * Additionally, make sure that package-reporter can report package
+ information to the server properly.
+
+ * Create a new, fresh container of the same series, install the current
+ version of Landscape Client from the main Ubuntu repo for that series
+ and add the benchmarking like before. Do not install lsb-core as it will
+ crash package-reporter before the benchmark runs. Make sure the
+ benchmarking is in place before continuing.
+
+ * sudo apt update && sudo apt install -y landscape-client
+ python3-psutil
+
+ * Connect this container to the same server and wait for package
+ reporter to run, and compare the benchmarking results with the proposed
+ version:
+
+ * less /var/lib/landscape/client/result.txt
+
+ * The cpu times reported by the proposed version should be
+ significantly less than those of the current version. With these
+ changes, this benchmark ran about 11x faster on Jammy and 15x faster on
+ Focal in my testing. See my full observations in the Other Info section.
+
+ [ Where problems could occur ]
+
+ * Because the fix also changed the approach to read the os-release file
+ instead of the lsb_release command output, this could introduce
+ unforeseen behavior with code that was written with the original
+ implementation in mind.
+
+ * Further, as this approach tweaks how package changes are computed,
+ the accuracy/reliability could be compromised. Further, this could lead
+ to issues with Landscape Client reporting package information too
+ frequently or at all.
+
+ * This proposed fix gets the archives directly from each package
+ versions rather than getting thing information from the Origin class of
+ the Python apt package. However, the point of this is to check if the
+ given package version matches the format of the security or backport
+ archives, so it's possible that this change could lead to package
+ version candidates being incorrectly skipped/not skipped.
+
+ * However, without these fixes, Landscape Client cannot report package
+ data with lsb modules installed so it is hard to imagine how this would
+ be worse. The performance gains also seem to be quite significant.
+
+ [ Other Info ]
+
+ * Benchmarking observations:
+ https://docs.google.com/spreadsheets/d/1AOGhskitsj5XbgJGg0Ab-
+ EcfG4GxcB8wjRnWX0wMefE
+
+ * Raw paste for the results from the current version (control) of
+ Landscape Client on Focal: https://pastebin.com/y04PE9rw
+
+ * Raw paste for the results from the modified (proposed) of Landscape
+ Client on Focal: https://pastebin.com/1cbHVGQR
+
+ * Raw paste for the results from the current version (control) of
+ Landscape Client on Jammy: https://pastebin.com/X3d1M8BV
+
+ * Raw paste for the results from the modified (proposed) of Landscape
+ Client on Jammy: https://pastebin.com/TtXpqJDE
+
+ * Original LP question/discussion for "High CPU usage of
+ /usr/bin/landscape-package-reporter":
+ https://answers.launchpad.net/landscape-client/+question/403745
+
+ * GH PR for the improvement to the performance of package-reporter:
+ https://github.com/canonical/landscape-client/pull/303
+
+ * GH PR for the fix for LP#2031036:
+ https://github.com/canonical/landscape-client/pull/170
+
+
+
+ Original bug report below:
+
+ --------
+
+
+ Package reporter from landscape client is failing to report to server because
seemingly of a lsb parse issue.
/var/log/landscape/package-reporter.log
2023-08-10 13:42:00,414 ERROR [MainThread] None
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653,
in _runCallbacks
current.result = callback(current.result, *args, **kw)
File
"/usr/lib/python3/dist-packages/landscape/client/package/taskhandler.py", line
217, in got_server_uuid
lsb_release_info = parse_lsb_release(self.lsb_release_filename)
File "/usr/lib/python3/dist-packages/landscape/lib/lsb_release.py", line
31, in parse_lsb_release
dist, desc, release, code_name, _ = lsb_info.decode().split("\n")
ValueError: too many values to unpack (expected 5)
Multiple LSB Packages are installed
/usr/bin/lsb_release
LSB Version:
core-9.20170808ubuntu1-noarch:printing-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch
lsb_release -a
LSB Version:
core-9.20170808ubuntu1-noarch:printing-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic
** Also affects: landscape-client (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2031036
Title:
Error when parsing lsb_release, landscape-package-reporter unable to
report package list to server
To manage notifications about this bug go to:
https://bugs.launchpad.net/landscape-client/+bug/2031036/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
