nxsbi commented on issue #8681: URL: https://github.com/apache/cloudstack/issues/8681#issuecomment-1998647793
Hello @kiranchavala Finally got back to this.. I logged into the control node. NOTE I am on 4.17.0 - is there any SystemVM level changes due to which this is not working-- kubectl get nodes (same message for kubectl get pods --all-namespaces) ``` root@K120-control-18e3f28025e:/opt/cloud/bin# kubectl get nodes E0314 23:22:12.988514 197290 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused E0314 23:22:12.989740 197290 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused E0314 23:22:12.990727 197290 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused E0314 23:22:12.993205 197290 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused E0314 23:22:12.994831 197290 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused The connection to the server localhost:8080 was refused - did you specify the right host or port? root@K120-control-18e3f28025e:/opt/cloud/bin# ``` So after this, I looked into the /var/log/daemon.log - which shows meaningful error messages. Relevant content from - /var/log/daemon.log below. However, I have no clue how to fix this. I did Google searches, and saw this https://github.com/containerd/containerd/discussions/8033 - which asks to check /etc/containerd/config.toml if cri is disabled - however it is not in the disabled list. ``` Mar 14 23:18:56 systemvm systemd[1]: deploy-kube-system.service: Scheduled restart job, restart counter is at 2094. Mar 14 23:18:56 systemvm systemd[1]: Stopped deploy-kube-system.service. Mar 14 23:18:56 systemvm systemd[1]: Started deploy-kube-system.service. Mar 14 23:18:56 systemvm deploy-kube-system[175532]: W0314 23:18:56.560147 175532 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration! Mar 14 23:18:56 systemvm deploy-kube-system[175532]: [init] Using Kubernetes version: v1.27.8 Mar 14 23:18:56 systemvm deploy-kube-system[175532]: [preflight] Running pre-flight checks Mar 14 23:18:56 systemvm deploy-kube-system[175532]: error execution phase preflight: [preflight] Some fatal errors occurred: Mar 14 23:18:56 systemvm deploy-kube-system[175532]: #011[ERROR CRI]: container runtime is not running: output: time="2024-03-14T23:18:56Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" Mar 14 23:18:56 systemvm deploy-kube-system[175532]: , error: exit status 1 Mar 14 23:18:56 systemvm deploy-kube-system[175532]: [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` Mar 14 23:18:56 systemvm deploy-kube-system[175532]: To see the stack trace of this error execute with --v=5 or higher Mar 14 23:18:56 systemvm deploy-kube-system[175560]: W0314 23:18:56.727208 175560 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration! Mar 14 23:18:56 systemvm deploy-kube-system[175560]: [init] Using Kubernetes version: v1.27.8 Mar 14 23:18:56 systemvm deploy-kube-system[175560]: [preflight] Running pre-flight checks Mar 14 23:18:56 systemvm deploy-kube-system[175560]: error execution phase preflight: [preflight] Some fatal errors occurred: Mar 14 23:18:56 systemvm deploy-kube-system[175560]: #011[ERROR CRI]: container runtime is not running: output: time="2024-03-14T23:18:56Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" Mar 14 23:18:56 systemvm deploy-kube-system[175560]: , error: exit status 1 Mar 14 23:18:56 systemvm deploy-kube-system[175560]: [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` Mar 14 23:18:56 systemvm deploy-kube-system[175560]: To see the stack trace of this error execute with --v=5 or higher Mar 14 23:18:56 systemvm deploy-kube-system[175587]: W0314 23:18:56.893921 175587 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration! Mar 14 23:18:56 systemvm deploy-kube-system[175587]: [init] Using Kubernetes version: v1.27.8 Mar 14 23:18:56 systemvm deploy-kube-system[175587]: [preflight] Running pre-flight checks Mar 14 23:18:57 systemvm deploy-kube-system[175587]: error execution phase preflight: [preflight] Some fatal errors occurred: Mar 14 23:18:57 systemvm deploy-kube-system[175587]: #011[ERROR CRI]: container runtime is not running: output: time="2024-03-14T23:18:56Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" Mar 14 23:18:57 systemvm deploy-kube-system[175587]: , error: exit status 1 Mar 14 23:18:57 systemvm deploy-kube-system[175587]: [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` Mar 14 23:18:57 systemvm deploy-kube-system[175587]: To see the stack trace of this error execute with --v=5 or higher Mar 14 23:18:57 systemvm deploy-kube-system[175531]: Error: kubeadm init failed! Mar 14 23:18:57 systemvm systemd[1]: deploy-kube-system.service: Main process exited, code=exited, status=1/FAILURE Mar 14 23:18:57 systemvm systemd[1]: deploy-kube-system.service: Failed with result 'exit-code'. Mar 14 23:18:57 systemvm systemd[1]: deploy-kube-system.service: Scheduled restart job, restart counter is at 2095. Mar 14 23:18:57 systemvm systemd[1]: Stopped deploy-kube-system.service. Mar 14 23:18:57 systemvm systemd[1]: Started deploy-kube-system.service. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
