Package: cloud.debian.org Severity: normal Certain functionality built in to cloud-init depends on a reasonably accurate clock, such as apt repo metadata signature verification. In the case where a system's hardware clock is far out of sync, chrony may not have completed synchronization before apt tries to refresh apt caches in the package_update_upgrade_install module or in a userdata shell script. An example of the resulting failure follows:
MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^? 169.254.169.123 3 6 1 1 -2986h[ -2986h] +/- 586us Sun Aug 7 13:25:16 UTC 2022 Get:2 http://cdn-aws.deb.debian.org/debian bullseye InRelease [116 kB] Get:3 http://cdn-aws.deb.debian.org/debian bullseye-updates InRelease [44.1 kB] Get:4 http://cdn-aws.deb.debian.org/debian bullseye-backports InRelease [49.0 kB] Reading package lists... E: Release file for http://security.debian.org/debian-security/dists/bullseye-security/InRelease is not valid yet (invalid for another 124d 5h 58min 50s). Updates for this repository will not be applied. E: Release file for http://cdn-aws.deb.debian.org/debian/dists/bullseye/InRelease is not valid yet (invalid for another 33d 20h 52min 34s). Updates for this repository will not be applied. E: Release file for http://cdn-aws.deb.debian.org/debian/dists/bullseye-updates/InRelease is not valid yet (invalid for another 124d 6h 53min 11s). Updates for this repository will not be applied. E: Release file for http://cdn-aws.deb.debian.org/debian/dists/bullseye-backports/InRelease is not valid yet (invalid for another 124d 6h 53min 11s). Updates for this repository will not be applied. 2022-08-07 13:25:17,663 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts) 2022-08-07 13:25:17,664 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed Cloud-init v. 20.4.1 finished at Sun, 07 Aug 2022 13:25:17 +0000. Datasource DataSourceEc2Local. Up 10.28 seconds The userdata script was: #!/bin/bash chronyc sources date DEBIAN_FRONTEND=noninteractive apt-get update A possible workaround to this is to run a command like `chronyc waitsync` before apt. It turns out to be not entirely trivial to insert such a command in the right place with userdata, though. We should make some reasonable effort to ensure the clock is in sync before executing the cloud-final.service, which is where all the package installation work happens. It's not immediately clear exactly where we should do this, whether it's in the chrony configs, cloud-init itself, cloud-init's systemd configuration, or elsewhere. And then, should the fix be in one of the packages or the images?
