GitHub user rafaelpagliari added a comment to the discussion: Upgrading K8s Cluster Failed
I'm only looking at the logs below; at first glance, I don't see an error that would explain exactly why this is happening. For clarification, the tests on both ACS are with the clusters at zero. That is, I simply created the cluster and sent it to upgrade both. Immediately after the log below, the cluster enters alert status in ACS, and I receive the error. These logs are repeated many times afterward, so I believe there is no direct correlation. Jul 30 16:36:21 systemvm kubelet[5482]: I0730 16:36:21.218040 5482 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/weave-net-szlwg" podStartSLOduration=5.217763853 podCreationTimestamp="2025-07-30 16:36:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-07-30 16:36:20.231952365 +0000 UTC m=+82.846380214" watchObservedRunningTime="2025-07-30 16:36:21.217763853 +0000 UTC m=+83.832191726" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.665646046Z" level=info msg="StopPodSandbox for \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\"" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.666075939Z" level=info msg="TearDown network for sandbox \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\" successfully" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.666220832Z" level=info msg="StopPodSandbox for \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\" returns successfully" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.666982916Z" level=info msg="RemovePodSandbox for \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\"" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.667049075Z" level=info msg="Forcibly stopping sandbox \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\"" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.667230536Z" level=info msg="TearDown network for sandbox \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\" successfully" Jul 30 16:36:57 systemvm containerd[5348]: time="2025-07-30T16:36:57.675659342Z" level=info msg="RemovePodSandbox \"7e6e4a443c74cf7f00e0f79c8aa4b4e6f194abf8ca0aad013a1d8da0bc8134c3\" returns successfully" Jul 30 16:37:28 systemvm systemd[1]: Starting User Manager for UID 1000... Jul 30 16:37:28 systemvm systemd[7907]: PAM failed: Critical error - immediate abort Jul 30 16:37:28 systemvm systemd[7907]: user@1000.service: Failed to set up PAM session: Operation not permitted Jul 30 16:37:28 systemvm systemd[7907]: user@1000.service: Failed at step PAM spawning /lib/systemd/systemd: Operation not permitted Jul 30 16:37:28 systemvm systemd[1]: user@1000.service: Main process exited, code=exited, status=224/PAM Jul 30 16:37:28 systemvm systemd[1]: user@1000.service: Failed with result 'exit-code'. Jul 30 16:37:28 systemvm systemd[1]: Failed to start User Manager for UID 1000. Jul 30 16:37:28 systemvm systemd[1]: Started Session 33 of user cloud. Jul 30 16:37:28 systemvm systemd[1]: session-33.scope: Succeeded. Jul 30 16:37:29 systemvm systemd[1]: Starting User Manager for UID 1000... Jul 30 16:37:29 systemvm systemd[7928]: PAM failed: Critical error - immediate abort Jul 30 16:37:29 systemvm systemd[7928]: user@1000.service: Failed to set up PAM session: Operation not permitted Jul 30 16:37:29 systemvm systemd[7928]: user@1000.service: Failed at step PAM spawning /lib/systemd/systemd: Operation not permitted Jul 30 16:37:29 systemvm systemd[1]: user@1000.service: Main process exited, code=exited, status=224/PAM Jul 30 16:37:29 systemvm systemd[1]: user@1000.service: Failed with result 'exit-code'. Jul 30 16:37:29 systemvm systemd[1]: Failed to start User Manager for UID 1000. Jul 30 16:37:29 systemvm systemd[1]: Started Session 34 of user cloud. Jul 30 16:37:29 systemvm systemd[1]: session-34.scope: Succeeded. GitHub link: https://github.com/apache/cloudstack/discussions/11006#discussioncomment-13936688 ---- This is an automatically sent email for users@cloudstack.apache.org. To unsubscribe, please send an email to: users-unsubscr...@cloudstack.apache.org
