** Description changed: [ Impact ] This has been affecting minimal and base pipelines for cloud images for - both oracular and plucky. The oracular and plucky images fail lxd - related tests because lxd installation cannot complete with the - following error `ConnectionResetError: [Errno 104] Connection reset by - peer`. + both oracular and plucky and blocking publication for these images. The + oracular and plucky images fail lxd related tests because lxd + installation cannot complete with the following error + `ConnectionResetError: [Errno 104] Connection reset by peer`. This error + is intermittent. The proposed upload allows retries of the installation to mitigate availability issues from the snapstore. [ Test Plan ] To reproduce the bug you can launch a container from either oracular or noble like so: `lxc init ubuntu-<SUITE>-daily:<SUITE> test && lxc start test && lxc exec test bash` where SUITE=oracular or noble. Upon connecting to the container you can run `lxd init --auto --storage- backend dir` which results into the error `ConnectionResetError: [Errno - 104] Connection reset by peer`. From journalctl lxd logs, the following - appears: `error: cannot install "lxd": Post + 104] Connection reset by peer` intermittently. From journalctl lxd logs, + the following appears: `error: cannot install "lxd": Post "https://api.snapcraft.io/v2/snaps/refresh":` The package with the proposed changes can be installed to allow the retry loop for installing lxd in the oracular/noble containers. [ Where problems could occur ] The possible effect of this change is posting to snapcraft more frequently which could increase the traffic to `https://api.snapcraft.io`. The number of retries is however limited and the installation failure intermittent so the additional retries should not have a significant impact on Snapcraft. The package itself does not have any changes as the retry targets only the installer. [ Other Info ] The same change in lxd-installer has been accepted for plucky and is has been in build pipelines for our plucky images for a few weeks now; these images do not fail on the lxd-installer anymore. [Original Description] Description: Ubuntu Plucky Puffin (development branch) (*LXD container) Release: 25.04 Image serial: 20250217 Source package: https://launchpad.net/ubuntu/plucky/+package/lxd-installer Package version: ``` $ apt-cache policy lxd-installer lxd-installer: Installed: 10 Candidate: 10 Version table: *** 10 100 100 /var/lib/dpkg/status ``` Expected behaviour: `lxd init --auto --storage-backend dir` command would trigger then `lxd-installer` shim and`lxd` would be installed in the container. What happened instead: ``` 10:19:41 2025-02-17 17:19:41,454 [INFO] test_framework.tests.lxd_start_stop: initializing lxd 10:19:41 Installing LXD snap, please be patient. 10:19:41 Traceback (most recent call last): 10:19:41 File "<string>", line 1, in <module> 10:19:41 import socket; s=socket.socket(socket.AF_UNIX); s.connect("/run/lxd-installer.socket"); s.send(b"x"); s.recv(1) 10:21:12 ~~~~~~^^^ 10:21:12 ConnectionResetError: [Errno 104] Connection reset by peer ``` The command that triggers the shim is `lxd init --auto --storage-backend dir` Reproducer: This failure is intermittent, and I have had a hard time getting a reliable reproducer for it. The best that I have got is: ``` lxc init ubuntu-minimal-daily:plucky test #20250217 serial was pulled lxc start test lxc exec test bash <inside the shell> lxd init --auto --storage-backend dir # confirm with 'y' ``` The error should appear almost instantly. If it does not, stop & delete the instance and repeat the above. *LXD team suggested reproducer: ``` $ while lxc launch ubuntu-minimal-daily:plucky test && lxc exec --force-noninteractive test -- lxc --version && lxc delete -f test; do sleep 1; done ``` Logs: `journalctl --grep lxd`: ``` root@genuine-satyr:~# journalctl --grep lxd Feb 25 23:35:52 genuine-satyr useradd[324]: add 'ubuntu' to group 'lxd' Feb 25 23:35:52 genuine-satyr useradd[324]: add 'ubuntu' to shadow group 'lxd' Feb 25 23:35:53 genuine-satyr systemd[1]: Starting lxd-installer.socket - Helper to install lxd snap on demand... Feb 25 23:35:53 genuine-satyr systemd[1]: Listening on lxd-installer.socket - Helper to install lxd snap on demand. Feb 25 23:35:56 genuine-satyr cloud-init[262]: Cloud-init v. 25.1~3geb1965a4-0ubuntu1 finished at Tue, 25 Feb 2025 23:35:56 +0000. Datasource DataSourceLXD. Up 5.51 seconds Feb 25 23:36:00 genuine-satyr systemd[1]: Created slice system-lxd\x2dinstaller.slice - Slice /system/lxd-installer. Feb 25 23:36:00 genuine-satyr systemd[1]: Started lxd-installer@0-503-0.service - Helper to install lxd snap on demand (PID 503/UID 0). Feb 25 23:36:00 genuine-satyr lxd-installer-service[504]: + [ lxd-installer-service = lxd-installer-service ] Feb 25 23:36:00 genuine-satyr lxd-installer-service[511]: + lxd_channel Feb 25 23:36:00 genuine-satyr lxd-installer-service[504]: + snap install lxd --channel=5.21/stable/ubuntu-25.04 Feb 25 23:36:00 genuine-satyr snapd[348]: api_snaps.go:467: Installing snap "lxd" revision unset Feb 25 23:36:00 genuine-satyr lxd-installer-service[512]: error: cannot install "lxd": Post "https://api.snapcraft.io/v2/snaps/refresh": Feb 25 23:36:00 genuine-satyr systemd[1]: lxd-installer@0-503-0.service: Main process exited, code=exited, status=1/FAILURE Feb 25 23:36:00 genuine-satyr systemd[1]: lxd-installer@0-503-0.service: Failed with result 'exit-code'. ``` `journalctl` around failure: ``` Feb 25 23:40:12 test systemd[1]: Created slice system-lxd\x2dinstaller.slice - Slice /system/lxd-installer. Feb 25 23:40:12 test systemd[1]: Started lxd-installer@0-508-0.service - Helper to install lxd snap on demand (PID 508/UID 0). Feb 25 23:40:12 test lxd-installer-service[509]: + [ lxd-installer-service = lxd-installer-service ] Feb 25 23:40:12 test lxd-installer-service[509]: + snap wait system seed.loaded Feb 25 23:40:12 test lxd-installer-service[516]: + lxd_channel Feb 25 23:40:12 test lxd-installer-service[516]: + track= Feb 25 23:40:12 test lxd-installer-service[516]: + [ -r /etc/os-release ] Feb 25 23:40:12 test lxd-installer-service[516]: + . /etc/os-release Feb 25 23:40:12 test lxd-installer-service[516]: + PRETTY_NAME=Ubuntu Plucky Puffin (development branch) Feb 25 23:40:12 test lxd-installer-service[516]: + NAME=Ubuntu Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION_ID=25.04 Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION=25.04 (Plucky Puffin) Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION_CODENAME=plucky Feb 25 23:40:12 test lxd-installer-service[516]: + ID=ubuntu Feb 25 23:40:12 test lxd-installer-service[516]: + ID_LIKE=debian Feb 25 23:40:12 test lxd-installer-service[516]: + HOME_URL=https://www.ubuntu.com/ Feb 25 23:40:12 test lxd-installer-service[516]: + SUPPORT_URL=https://help.ubuntu.com/ Feb 25 23:40:12 test lxd-installer-service[516]: + BUG_REPORT_URL=https://bugs.launchpad.net/ubuntu/ Feb 25 23:40:12 test lxd-installer-service[516]: + PRIVACY_POLICY_URL=https://www.ubuntu.com/legal/terms-and-policies/privacy-policy Feb 25 23:40:12 test lxd-installer-service[516]: + UBUNTU_CODENAME=plucky Feb 25 23:40:12 test lxd-installer-service[516]: + LOGO=ubuntu-logo Feb 25 23:40:12 test lxd-installer-service[516]: + track=5.21 Feb 25 23:40:12 test lxd-installer-service[516]: + [ -n 5.21 ] Feb 25 23:40:12 test lxd-installer-service[516]: + [ -n 25.04 ] Feb 25 23:40:12 test lxd-installer-service[516]: + echo 5.21/stable/ubuntu-25.04 Feb 25 23:40:12 test lxd-installer-service[509]: + CHANNEL=5.21/stable/ubuntu-25.04 Feb 25 23:40:12 test lxd-installer-service[509]: + [ -z 5.21/stable/ubuntu-25.04 ] Feb 25 23:40:12 test lxd-installer-service[509]: + snap install lxd --channel=5.21/stable/ubuntu-25.04 Feb 25 23:40:12 test snapd[345]: api_snaps.go:467: Installing snap "lxd" revision unset Feb 25 23:40:12 test snapd[345]: store_download.go:142: no host system xdelta3 available to use deltas Feb 25 23:40:12 test lxd-installer-service[517]: error: cannot install "lxd": Post "https://api.snapcraft.io/v2/snaps/refresh": Feb 25 23:40:12 test lxd-installer-service[517]: context canceled Feb 25 23:40:12 test systemd[1]: lxd-installer@0-508-0.service: Main process exited, code=exited, status=1/FAILURE Feb 25 23:40:12 test systemd[1]: lxd-installer@0-508-0.service: Failed with result 'exit-code'. Feb 25 23:40:14 test snapd[345]: overlord.go:518: Released state lock file Feb 25 23:40:14 test snapd[345]: daemon stop requested to wait for socket activation Feb 25 23:40:14 test systemd[1]: snapd.service: Deactivated successfully. Feb 25 23:40:14 test systemd[1]: snapd.service: Consumed 1.014s CPU time, 37.6M memory peak. ```
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2100564 Title: lxd-installer shim fails to install with snapstore error To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lxd-installer/+bug/2100564/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
