I've made some measurements on the last 12 hours of running tests, the
proportion of upstream tests seems appropriate, so I'll mark this as
"Fix released"
** Changed in: auto-package-testing
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/2062537
Title:
Change the way upstream jobs percentage is done
Status in Auto Package Testing:
Fix Released
Bug description:
This code[1] is mostly broken: it's only when connecting to the queues that
we decide if a worker is going to take `upstream` jobs, but this is based on
randomness. I see the following flaws:
* with low number of workers (20 is low), there is a high chance that the
distribution will not be uniform at all, meaning you can end up with far more
or far less workers taking upstream jobs than intended.
* the dice is rolled only when the worker starts, and that means that is you
end up with too many/few workers taking upstream jobs, the situation is going
to stay until the workers are restarted.
Here is a quick Python one-liner to play with. It gives the number of workers
that will pick up upstream jobs, over a total of 20, with a chance of 50%:
`len([r for r in [random.randint(1, 100) for _ in range(20)] if r > 50])`
For only 20 workers and 50% chances, it's common to see a run with a
deviation of more than 3 points from the objective (objective: 10, we often see
<=7 or >=13). This gets worse when setting a lower threshold (eg 15), because
we more often hit the extreme where no worker at all would pick upstream jobs.
A better solution would be to roll the dice in the `request` function
callback [2] so that for every upstream test request, the chance of processing
get calculated. This will make the number of rolls proportionate with the
number of jobs processed, and that means far greater numbers, which means far
more accurate percentage of jobs filtered by the threshold over time.
[1]:
https://git.launchpad.net/autopkgtest-cloud/tree/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker#n1482
[2]: around here:
https://git.launchpad.net/autopkgtest-cloud/tree/charms/focal/autopkgtest-cloud-worker/autopkgtest-cloud/worker/worker#n640
To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/2062537/+subscriptions
--
Mailing list: https://launchpad.net/~canonical-ubuntu-qa
Post to : canonical-ubuntu-qa@lists.launchpad.net
Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa
More help : https://help.launchpad.net/ListHelp
