I spend a bit of quality time with this bug today and it seems there is (also?)
a kernel dimension to it. I ran the following:
"""
$ git status
On branch tests-use-core18-from-gce
Your branch is up to date with 'sergiocazzolato/tests-use-core18-from-gce'.
$ git diff
diff --git a/overlord/devicestate/devicestate.go
b/overlord/devicestate/devicestate.go
index d40b32bcbd..77ec3fc0a8 100644
--- a/overlord/devicestate/devicestate.go
+++ b/overlord/devicestate/devicestate.go
@@ -117,6 +117,8 @@ func canAutoRefresh(st *state.State) (bool, error) {
if !seeded {
return false, nil
}
+ // HACK
+ return false, nil
// Try to ensure we have an accurate time before doing any
// refreshy stuff. Note that this call will not block.
diff --git a/spread.yaml b/spread.yaml
index 0aef5dea7c..68cbdc09b7 100644
--- a/spread.yaml
+++ b/spread.yaml
@@ -37,7 +37,7 @@ environment:
MANAGED_DEVICE: "false"
# a global setting for LXD channel to use in the tests
LXD_SNAP_CHANNEL: "latest/edge"
- UBUNTU_IMAGE_SNAP_CHANNEL: "latest/candidate"
+ UBUNTU_IMAGE_SNAP_CHANNEL: "beta/1.11"
CORE_CHANNEL: '$(HOST: echo "${SPREAD_CORE_CHANNEL:-edge}")'
BASE_CHANNEL: '$(HOST: echo "${SPREAD_BASE_CHANNEL:-edge}")'
KERNEL_CHANNEL: '$(HOST: echo "${SPREAD_KERNEL_CHANNEL:-edge}")'
diff --git a/tests/lib/prepare.sh b/tests/lib/prepare.sh
index e6b984c4d0..215d0611be 100755
--- a/tests/lib/prepare.sh
+++ b/tests/lib/prepare.sh
@@ -973,8 +973,8 @@ EOF
fi
if os.query is-core18; then
- curl -s -o core18.snap
https://storage.googleapis.com/snapd-spread-tests/snaps/core18_20211102_amd64.snap
- EXTRA_FUNDAMENTAL="$EXTRA_FUNDAMENTAL --snap $PWD/core18.snap"
+ curl --insecure -o pc-kernel.snap
https://people.canonical.com/~mvo/tmp/pYVQrBcKmBa0mZ4CCN7ExT6jH8rY1hza_831.snap
+ EXTRA_FUNDAMENTAL="$EXTRA_FUNDAMENTAL --snap $PWD/pc-kernel.snap"
fi
/snap/bin/ubuntu-image snap \
"""
which essentially moves the image to a 2 month older 4.14 kernel. With that I
could not reproduce the error for three subsequent runs of 2 repeats.
However when I disabled this code and use the current "18/edge" pc-
kernel (or stable, it's the same revision) then I can trigger the bug
right away.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1949089
Title:
systemd randomly fails to activate mount units in Ubuntu Core 18
Status in systemd package in Ubuntu:
New
Status in systemd source package in Bionic:
New
Bug description:
Since a month or so, we've been seeing random failures in our snapd
spread tests where systemd could not start the mount unit associated
with a snap because of a failed dependency.
The issue is described in the comments to PR
https://github.com/snapcore/snapd/pull/10935, but I'll summarize it
here.
When starting a snap, snapd creates a mount unit to mount the snap's
squashfs (the template is
https://github.com/snapcore/snapd/blob/release/2.53/systemd/systemd.go#L1186-L1205).
The snapd asks systemd to reload the configuration, and starts the
mount unit.
The failure we've observed is that sometimes systemd decides to stop
our mount unit (search for "Unmounting Mount unit for test-snapd-svc-
flip-flop" in the attached log), and then tries to reactivate it
again, and at that point it fails.
When I asked for help, Lukas pointed out that the latest update
contains a patch that is related to reload handling and mount units:
http://launchpadlibrarian.net/555420796/systemd_237-3ubuntu10.51_237-3ubuntu10.52.diff.gz
(the patch itself is better visible at
https://github.com/systemd/systemd/commit/f0831ed2a03fcef582660be1c3b1a9f3e267e656).
When looking at the systemd git log, though, I noticed another patch
that was applied shortly after this one, which also seems related but
was not backported:
https://github.com/systemd/systemd/commit/04eb582acc203eab0bc5c2cc5e13986f16e09df0
Since the stopping of our mount unit happens immediately after a
systemd reload, it actually seems very likely that the inclusion of
f0831ed2a03fcef582660be1c3b1a9f3e267e656 in the systemd update is what
causes our woes (though, indeed, the issue is not reliably
reproducible, so we cannot be sure).
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1949089/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
