[Canonical-ubuntu-qa] [Merge] ~hyask/autopkgtest-cloud:skia/fix_upstream_percentage into autopkgtest-cloud:master
Skia has proposed merging ~hyask/autopkgtest-cloud:skia/fix_upstream_percentage
into autopkgtest-cloud:master.
Requested reviews:
Canonical's Ubuntu QA (canonical-ubuntu-qa)
Related bugs:
Bug #2062537 in Auto Package Testing: "Change the way upstream jobs
percentage is done"
https://bugs.launchpad.net/auto-package-testing/+bug/2062537
For more details, see:
https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/464721
Fixes https://bugs.launchpad.net/auto-package-testing/+bug/2062537
--
Your team Canonical's Ubuntu QA is requested to review the proposed merge of
~hyask/autopkgtest-cloud:skia/fix_upstream_percentage into
autopkgtest-cloud:master.
diff --git a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
index bfa35e7..53fa05b 100755
--- a/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
+++ b/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker
@@ -650,6 +650,19 @@ def request(msg):
systemd_logging_handler._extra["ADT_RELEASE"] = release
systemd_logging_handler._extra["ADT_ARCH"] = architecture
+if "-upstream-" in msg.delivery_info["routing_key"]:
+# crude way to not allow upstream tests to monopolise resources - only 20%
+# of chances to really run them
+if random.randint(1, 100) > 20:
+logging.info(
+"Upstream job detected and dice rolled above 20, skipping for now"
+)
+return
+else:
+logging.info(
+"Upstream job detected and dice rolled below 20, running it now"
+)
+
body = msg.body
if isinstance(body, bytes):
try:
@@ -1475,12 +1488,7 @@ def amqp_connect(cfg, callback):
# avoid preferring the same architecture on all workers
queues = []
-contexts = ["", "huge-", "ppa-"]
-
-# crude way to not allow upstream tests to monopolise resources - only 50%
-# of workers will take them
-if random.randint(1, 100) < 50:
-contexts += ["upstream-"]
+contexts = ["", "huge-", "ppa-", "upstream-"]
for release in cfg.get("autopkgtest", "releases").split():
for context in contexts:
--
Mailing list: https://launchpad.net/~canonical-ubuntu-qa
Post to : canonical-ubuntu-qa@lists.launchpad.net
Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa
More help : https://help.launchpad.net/ListHelp
[Canonical-ubuntu-qa] [Bug 2062537] Re: Change the way upstream jobs percentage is done
** Merge proposal linked: https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/464721 -- You received this bug notification because you are a member of Canonical's Ubuntu QA, which is subscribed to Auto Package Testing. https://bugs.launchpad.net/bugs/2062537 Title: Change the way upstream jobs percentage is done Status in Auto Package Testing: New Bug description: This code[1] is mostly broken: it's only when connecting to the queues that we decide if a worker is going to take `upstream` jobs, but this is based on randomness. I see the following flaws: * with low number of workers (20 is low), there is a high chance that the distribution will not be uniform at all, meaning you can end up with far more or far less workers taking upstream jobs than intended. * the dice is rolled only when the worker starts, and that means that is you end up with too many/few workers taking upstream jobs, the situation is going to stay until the workers are restarted. Here is a quick Python one-liner to play with. It gives the number of workers that will pick up upstream jobs, over a total of 20, with a chance of 50%: `len([r for r in [random.randint(1, 100) for _ in range(20)] if r > 50])` For only 20 workers and 50% chances, it's common to see a run with a deviation of more than 3 points from the objective (objective: 10, we often see <=7 or >=13). This gets worse when setting a lower threshold (eg 15), because we more often hit the extreme where no worker at all would pick upstream jobs. A better solution would be to roll the dice in the `request` function callback [2] so that for every upstream test request, the chance of processing get calculated. This will make the number of rolls proportionate with the number of jobs processed, and that means far greater numbers, which means far more accurate percentage of jobs filtered by the threshold over time. [1]: https://git.launchpad.net/autopkgtest-cloud/tree/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker#n1482 [2]: around here: https://git.launchpad.net/autopkgtest-cloud/tree/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker#n640 To manage notifications about this bug go to: https://bugs.launchpad.net/auto-package-testing/+bug/2062537/+subscriptions -- Mailing list: https://launchpad.net/~canonical-ubuntu-qa Post to : canonical-ubuntu-qa@lists.launchpad.net Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa More help : https://help.launchpad.net/ListHelp
[Canonical-ubuntu-qa] [Bug 2061107] Re: shiftfs-lxd.sh in ubuntu_overlayfs_smoke_test failed with "Required tool 'zpool' is missing"
Patch applied, failed test restarted. ** Changed in: ubuntu-kernel-tests Status: In Progress => Fix Released -- You received this bug notification because you are a member of Canonical Platform QA Team, which is subscribed to ubuntu-kernel-tests. https://bugs.launchpad.net/bugs/2061107 Title: shiftfs-lxd.sh in ubuntu_overlayfs_smoke_test failed with "Required tool 'zpool' is missing" Status in ubuntu-kernel-tests: Fix Released Status in lxd package in Ubuntu: Invalid Bug description: Issue found in cycle 2024.04.01 with F-gkeop (5.4.0-1091.95), F-oracle (5.4.0-1123.132) Test log Running 'sudo -iu ubuntu /home/ubuntu/autotest/client/tmp/ubuntu_overlayfs_smoke_test/src/overlay-shiftfs-tests/tests/shiftfs-lxd.sh' Error: Required tool 'zpool' is missing Creating c1 Error: Failed loading storage pool "zfs": Storage pool not found Error: Failed to fetch instance "c1" in project "default": Instance not found FAIL Error: Failed to fetch instance "c1" in project "default": Instance not found Error: Failed checking instance exists "local:c1": Failed to fetch instance "c1" in project "default": Instance not found Error: Storage pool not found This is the failing command in shiftfs-lxd.sh: $ sudo lxc storage create zfs zfs size=2GiB Error: Required tool 'zpool' is missing $ snap list lxd Name Version RevTracking Publisher Notes lxd 5.21.1-43998c6 28155 latest/stable canonical✓ - This issue does not exist in 2024.03.04 cycle for these two kernels. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2061107/+subscriptions -- Mailing list: https://launchpad.net/~canonical-ubuntu-qa Post to : canonical-ubuntu-qa@lists.launchpad.net Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa More help : https://help.launchpad.net/ListHelp
