Tested with 1.1ubuntu1.18.04.7~16.04.1: On the autopkgtest infrastructure u-u runs for 20s when all packages are installed from xenial-security but none from xenial-updates:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/u/unattended-upgrades/20181213_182038_2962e@/log.gz ... Starting unattended upgrades script Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial Packages that will be upgraded: 19.18user 1.17system 0:20.53elapsed 99%CPU (0avgtext+0avgdata 77720maxresident)k 0inputs+123512outputs (0major+38986minor)pagefaults 0swaps ... On a 2012 MacBook Air inside a KVM qemu vm autopkgtest runner it is ~8s: ... adt-2.log-Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial adt-2.log-Packages that will be upgraded: adt-2.log:7.64user 0.31system 0:08.00elapsed 99%CPU (0avgtext+0avgdata 76516maxresident)k .. This is basically the same speed as with 0.90ubuntu0.10. There is a 12% speed regression when testing in qemu with kvm on a 19.04 host: ... adt-1549534420.log-Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial adt-1549534420.log-Packages that will be upgraded: adt-1549534420.log:6.72user 0.30system 0:07.07elapsed 99%CPU (0avgtext+0avgdata 77812maxresident)k ... vs. .. adt-2.log-Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial adt-2.log-Packages that will be upgraded: adt-2.log:7.64user 0.31system 0:08.00elapsed 99%CPU (0avgtext+0avgdata 76516maxresident)k ... Or a 24% speedup with a smaller set of upgradable packages measured in two 16.04 lxc containers running on the same 19.04 development system: ii unattended-upgrades 0.90ubuntu0.10 all automatic installation of security upgrades # for i in $(seq 5); do time unattended-upgrade --dry-run; done real 0m4.326s user 0m4.245s sys 0m0.043s real 0m4.309s user 0m4.239s sys 0m0.070s ... # apt list --upgradable Listing... Done cloud-init/xenial-proposed 18.5-21-g8ee294d5-0ubuntu1~16.04.1 all [upgradable from: 18.4-0ubuntu1~16.04.2] kmod/xenial-proposed 22-1ubuntu5.2 amd64 [upgradable from: 22-1ubuntu5.1] libc-bin/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] libc6/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] libkmod2/xenial-proposed 22-1ubuntu5.2 amd64 [upgradable from: 22-1ubuntu5.1] locales/xenial-proposed 2.23-0ubuntu11 all [upgradable from: 2.23-0ubuntu10] multiarch-support/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] python-apt-common/xenial-proposed 1.1.0~beta1ubuntu0.16.04.3 all [upgradable from: 1.1.0~beta1ubuntu0.16.04.2] python3-apt/xenial-proposed 1.1.0~beta1ubuntu0.16.04.3 amd64 [upgradable from: 1.1.0~beta1ubuntu0.16.04.2] snapd/xenial-proposed 2.37.1 amd64 [upgradable from: 2.34.2] ubuntu-core-launcher/xenial-proposed 2.37.1 amd64 [upgradable from: 2.34.2] unattended-upgrades/xenial-proposed 1.1ubuntu1.18.04.7~16.04.1 all [upgradable from: 0.90ubuntu0.10] # vs. ii unattended-upgrades 1.1ubuntu1.18.04.7~16.04.1 all automatic installation of security upgrades # for i in $(seq 5); do time unattended-upgrade --dry-run; done real 0m3.269s user 0m3.194s sys 0m0.076s real 0m3.277s user 0m3.135s sys 0m0.115s ... root@x-uu-ref:~# apt list --upgradable Listing... Done cloud-init/xenial-proposed 18.5-21-g8ee294d5-0ubuntu1~16.04.1 all [upgradable from: 18.4-0ubuntu1~16.04.2] kmod/xenial-proposed 22-1ubuntu5.2 amd64 [upgradable from: 22-1ubuntu5.1] libc-bin/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] libc6/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] libkmod2/xenial-proposed 22-1ubuntu5.2 amd64 [upgradable from: 22-1ubuntu5.1] locales/xenial-proposed 2.23-0ubuntu11 all [upgradable from: 2.23-0ubuntu10] multiarch-support/xenial-proposed 2.23-0ubuntu11 amd64 [upgradable from: 2.23-0ubuntu10] python-apt-common/xenial-proposed 1.1.0~beta1ubuntu0.16.04.3 all [upgradable from: 1.1.0~beta1ubuntu0.16.04.2] snapd/xenial-proposed 2.37.1 amd64 [upgradable from: 2.34.2] ubuntu-core-launcher/xenial-proposed 2.37.1 amd64 [upgradable from: 2.34.2] root@x-uu-ref:~# ** Tags removed: verification-needed verification-needed-xenial ** Tags added: verification-done verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to unattended-upgrades in Ubuntu. https://bugs.launchpad.net/bugs/1396787 Title: checking trust of archives eats a lot of cpu Status in unattended-upgrades package in Ubuntu: Fix Released Status in unattended-upgrades source package in Xenial: Fix Committed Status in unattended-upgrades source package in Bionic: Fix Released Bug description: [Impact] * Unattended-upgrades consumes tens of seconds or even minutes of CPU time to verify the origin of the packages * Using excessive amount of CPU is unpleasant for desktop/laptop users and also wastes computation time on servers/cloud instances. * Unattended-upgrades' algorithm for checking and adjusting package origins is redesigned to visit and adjust less packages. [Test Case] * The added upgrade-all-security autopkgtest measure the time u-u needs for upgrading security updates on the tested release starting with no security updates applied to the point where all security updates are applied but all packages are left upgradable from <release>-updates. The test also measures the time needed for --dry-run to find no updates to be installed unattended. * Please run autopkgtests and look for the to time results: ... All upgrades installed 44.41user 3.06system 0:48.35elapsed 98%CPU (0avgtext+0avgdata 164872maxresident)k 208inputs+192376outputs (0major+642657minor)pagefaults 0swaps ... No packages found that can be upgraded unattended and no pending auto-removals 2.83user 0.11system 0:02.98elapsed 98%CPU (0avgtext+0avgdata 79308maxresident)k [Regression Potential] * Due to algorithm redesign there is a risk that packages from allowed origins are not upgraded. There were unit tests for testing the selection of the right packages to upgrade already, but a new autopkgtest is also introduce to verify u-u's behavior on current real-life security-updates. [Original bug text] (System: Ubuntu 14.04, up to date packages) I noticed that unattended-upgrades spends a significant amount of time in phases where it runs at 100% cpu. On a slower machine (core 2 t7200 2GHz) this goes on for minutes rather than seconds. This interferes with using the machine for other tasks. Using the --debug option to unattended-upgrades shows that the program outputs a lot of lines like the following during these 100% cpu phases: matching 'a'='trusty-updates' against '<Origin component:'universe' archive:'trusty-updates' origin:'Ubuntu' label:'Ubuntu' site:'de.archive.ubuntu.com' isTrusted:True> From this output I guess the operation executed is not so complicated that it should require so much cpu power. ?? ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: unattended-upgrades 0.82.1ubuntu2 ProcVersionSignature: Ubuntu 3.13.0-40.69-generic 3.13.11.10 Uname: Linux 3.13.0-40-generic x86_64 ApportVersion: 2.14.1-0ubuntu3.5 Architecture: amd64 Date: Wed Nov 26 21:53:57 2014 InstallationDate: Installed on 2014-08-28 (90 days ago) InstallationMedia: Kubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140416.1) PackageArchitecture: all ProcEnviron: TERM=xterm PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=de_DE.UTF-8 SHELL=/bin/bash SourcePackage: unattended-upgrades UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1396787/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
