tomorrrow666 opened a new issue, #10426: URL: https://github.com/apache/cloudstack/issues/10426
idrl@management1:~$ sudo tail -f /var/log/cloudstack/agent/agent.log 2025-02-19 10:33:21,758 INFO [cloud.agent.AgentShell] (main:[]) (logid:) Using default Java settings for IPv6 preference for agent connection 2025-02-19 10:33:21,823 INFO [cloud.agent.Agent] (main:[]) (logid:) id is 2025-02-19 10:33:21,829 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) uefi.properties file found at /etc/cloudstack/agent/uefi.properties 2025-02-19 10:33:21,829 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) guest.nvram.template.legacy = null 2025-02-19 10:33:21,829 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) guest.loader.legacy = null 2025-02-19 10:33:21,829 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) guest.nvram.template.secure = null 2025-02-19 10:33:21,830 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) guest.loader.secure =null 2025-02-19 10:33:21,830 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) guest.nvram.path = null 2025-02-19 10:33:21,882 INFO [kvm.resource.LibvirtConnection] (main:[]) (logid:) No existing libvirtd connection found. Opening a new one 2025-02-19 10:33:22,027 INFO [kvm.resource.LibvirtComputingResource] (main:[]) (logid:) IO uring driver for Qemu: enabled idrl@management1:~$ sudo tail -f /var/log/cloudstack/agent/agent.err at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:453) at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:432) at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:416) at com.cloud.agent.AgentShell.start(AgentShell.java:512) at com.cloud.agent.AgentShell.main(AgentShell.java:542) Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ... 14 more idrl@management1:~$ sudo tail -f /var/log/cloudstack/management/management-server.log 2025-02-19 10:36:13,976 DEBUG [c.c.s.S.AutoScaleMonitor] (StatsCollector-3:[ctx-2e1a9507]) (logid:bac1a6d1) AutoScaling Monitor is running... 2025-02-19 10:36:13,978 DEBUG [c.c.v.d.VmStatsDaoImpl] (StatsCollector-4:[ctx-4e3079a0]) (logid:92b0a506) Starting to remove all vm_stats rows older than [Tue Feb 18 22:36:13 GMT+08:00 2025]. 2025-02-19 10:36:13,981 DEBUG [c.c.s.d.VolumeStatsDaoImpl] (StatsCollector-6:[ctx-a93b8a32]) (logid:65fd1ad6) Starting to remove all volume_stats rows older than [Tue Feb 18 22:36:13 GMT+08:00 2025]. 2025-02-19 10:36:13,981 INFO [c.c.v.d.VmStatsDaoImpl] (StatsCollector-4:[ctx-4e3079a0]) (logid:92b0a506) Removed a total of [0] vm_stats rows older than [Tue Feb 18 22:36:13 GMT+08:00 2025]. 2025-02-19 10:36:13,983 INFO [c.c.s.d.VolumeStatsDaoImpl] (StatsCollector-6:[ctx-a93b8a32]) (logid:65fd1ad6) Removed a total of [0] volume_stats rows older than [Tue Feb 18 22:36:13 GMT+08:00 2025]. 2025-02-19 10:36:14,438 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2deeed7d]) (logid:afa45fb2) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:14,439 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2deeed7d]) (logid:afa45fb2) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:14,458 DEBUG [c.c.a.ApiServer] (qtp1215904751-317:[ctx-2deeed7d, ctx-30259112]) (logid:afa45fb2) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:14,466 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-317:[ctx-2deeed7d, ctx-30259112]) (logid:afa45fb2) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:14,466 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-317:[ctx-2deeed7d, ctx-30259112]) (logid:afa45fb2) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:14,471 INFO [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2deeed7d, ctx-30259112]) (logid:afa45fb2) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:14,471 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2deeed7d, ctx-30259112]) (logid:afa45fb2) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:16,423 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-3:[ctx-55f241df]) (logid:d87ff959) HA health check task is running... 2025-02-19 10:36:19,435 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-e386a4f1]) (logid:85e19109) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:19,435 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-e386a4f1]) (logid:85e19109) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:19,438 DEBUG [c.c.a.ApiServer] (qtp1215904751-20:[ctx-e386a4f1, ctx-2a2ab2ee]) (logid:85e19109) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:19,439 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-20:[ctx-e386a4f1, ctx-2a2ab2ee]) (logid:85e19109) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:19,439 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-20:[ctx-e386a4f1, ctx-2a2ab2ee]) (logid:85e19109) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:19,440 INFO [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-e386a4f1, ctx-2a2ab2ee]) (logid:85e19109) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:19,441 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-e386a4f1, ctx-2a2ab2ee]) (logid:85e19109) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:20,426 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-4:[ctx-f04c5bb0]) (logid:34b96efe) HA health check task is running... 2025-02-19 10:36:22,604 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-21f7bee8]) (logid:87ae5750) Enabled non-edge zones available for scan: 1 2025-02-19 10:36:22,605 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-21f7bee8]) (logid:87ae5750) Zone 1 has no host available which is enabled and in Up state 2025-02-19 10:36:22,605 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-21f7bee8]) (logid:87ae5750) Zone [1] is not ready to launch secondary storage VM. 2025-02-19 10:36:22,623 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-8cfb49af]) (logid:ba3842b2) Skip capacity scan as there is no Primary Storage in 'Up' state 2025-02-19 10:36:24,428 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-4:[ctx-a17df001]) (logid:96a678aa) HA health check task is running... 2025-02-19 10:36:24,439 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-69b2a6b3]) (logid:53e50c67) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:24,439 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-69b2a6b3]) (logid:53e50c67) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:24,455 DEBUG [c.c.a.ApiServer] (qtp1215904751-317:[ctx-69b2a6b3, ctx-0b1c9fe5]) (logid:53e50c67) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:24,460 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-317:[ctx-69b2a6b3, ctx-0b1c9fe5]) (logid:53e50c67) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:24,460 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-317:[ctx-69b2a6b3, ctx-0b1c9fe5]) (logid:53e50c67) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:24,466 INFO [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-69b2a6b3, ctx-0b1c9fe5]) (logid:53e50c67) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:24,467 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-69b2a6b3, ctx-0b1c9fe5]) (logid:53e50c67) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:28,433 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-4:[ctx-8c21398f]) (logid:b42e03f6) HA health check task is running... 2025-02-19 10:36:28,994 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-3896991a]) (logid:c1e7ca4f) Found 0 routers to update status. 2025-02-19 10:36:28,996 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-3896991a]) (logid:c1e7ca4f) Found 0 VPC's to update Redundant State. 2025-02-19 10:36:28,996 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-3896991a]) (logid:c1e7ca4f) Found 0 networks to update RvR status. 2025-02-19 10:36:29,441 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-8d4fd277]) (logid:636df5fc) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:29,441 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-8d4fd277]) (logid:636df5fc) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:29,446 DEBUG [c.c.a.ApiServer] (qtp1215904751-20:[ctx-8d4fd277, ctx-1c517bce]) (logid:636df5fc) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:29,453 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-20:[ctx-8d4fd277, ctx-1c517bce]) (logid:636df5fc) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:29,453 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-20:[ctx-8d4fd277, ctx-1c517bce]) (logid:636df5fc) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:29,463 INFO [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-8d4fd277, ctx-1c517bce]) (logid:636df5fc) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:29,463 DEBUG [c.c.a.ApiServlet] (qtp1215904751-20:[ctx-8d4fd277, ctx-1c517bce]) (logid:636df5fc) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:32,435 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-2:[ctx-05b17f2f]) (logid:82f7fca8) HA health check task is running... 2025-02-19 10:36:34,439 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-23c45e9a]) (logid:dd98e14c) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:34,440 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-23c45e9a]) (logid:dd98e14c) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:34,446 DEBUG [c.c.a.ApiServer] (qtp1215904751-317:[ctx-23c45e9a, ctx-5cca524d]) (logid:dd98e14c) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:34,449 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-317:[ctx-23c45e9a, ctx-5cca524d]) (logid:dd98e14c) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:34,449 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-317:[ctx-23c45e9a, ctx-5cca524d]) (logid:dd98e14c) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:34,453 INFO [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-23c45e9a, ctx-5cca524d]) (logid:dd98e14c) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:34,453 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-23c45e9a, ctx-5cca524d]) (logid:dd98e14c) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:35,912 DEBUG [c.c.a.ApiServlet] (qtp1215904751-15:[ctx-be65162f]) (logid:accee293) ===START=== 192.168.1.199 -- POST command=addHost&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:35,912 DEBUG [c.c.a.ApiServlet] (qtp1215904751-15:[ctx-be65162f]) (logid:accee293) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:35,915 DEBUG [c.c.a.ApiServer] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:35,917 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:35,917 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:35,920 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Received unknown parameters for command addHost. Unknown parameters : clustertype 2025-02-19 10:36:35,929 INFO [c.c.r.ResourceManagerImpl] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Trying to add a new host at http://192.168.1.20 in data center 1 2025-02-19 10:36:36,122 WARN [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Failed to authenticate with ssh key 2025-02-19 10:36:36,122 INFO [c.c.h.k.d.KvmServerDiscoverer] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Failed to authenticate with ssh key, retrying with password 2025-02-19 10:36:36,180 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Executing cmd: ls /dev/kvm 2025-02-19 10:36:36,437 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-2:[ctx-65800cb4]) (logid:be62768c) HA health check task is running... 2025-02-19 10:36:37,424 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) SSH command: ls /dev/kvm SSH command output:/dev/kvm 2025-02-19 10:36:37,454 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Executing cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/ 2025-02-19 10:36:39,436 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-2cbc03ab]) (logid:35117a38) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:39,438 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-2cbc03ab]) (logid:35117a38) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:39,444 DEBUG [c.c.a.ApiServer] (qtp1215904751-17:[ctx-2cbc03ab, ctx-adc918c6]) (logid:35117a38) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:39,445 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-17:[ctx-2cbc03ab, ctx-adc918c6]) (logid:35117a38) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:39,445 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-17:[ctx-2cbc03ab, ctx-adc918c6]) (logid:35117a38) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:39,448 INFO [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-2cbc03ab, ctx-adc918c6]) (logid:35117a38) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:39,448 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-2cbc03ab, ctx-adc918c6]) (logid:35117a38) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:39,945 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) SSH command: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/ SSH command output: 2025-02-19 10:36:39,984 WARN [o.a.c.m.w.WebhookServiceImpl] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Skipping delivering event [ID: null, description: {"details":"domain(s): [192.168.1.20, 192.168.1.20] addresses: [192.168.1.20]","event":"CA.CERTIFICATE.ISSUE","status":"Completed"}] to any webhook as account ID is missing 2025-02-19 10:36:39,984 WARN [o.a.c.f.e.EventDistributorImpl] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Failed to publish event [category: ActionEvent, type: CA.CERTIFICATE.ISSUE] on bus webhookEventBus 2025-02-19 10:36:40,000 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Executing cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-cert-import /etc/cloudstack/agent/agent.properties PVUVJb68aTYwEFBD /etc/cloudstack/agent/ 2025-02-19 10:36:40,439 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-4:[ctx-30d56589]) (logid:d51699a0) HA health check task is running... 2025-02-19 10:36:42,637 DEBUG [c.c.h.k.d.KvmServerDiscoverer] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Succeeded to import certificate in the keystore for agent on the KVM host: 192.168.1.20. Agent secured and trusted. 2025-02-19 10:36:42,644 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) Executing cmd: cloudstack-setup-agent -m 192.168.1.20 -z 1 -p 1 -c 1 -g 7eb7429b-bf42-33cf-bc97-03f4d3d2c424 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm 2025-02-19 10:36:44,427 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-14f0eb9d]) (logid:8f9443c5) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:44,427 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-14f0eb9d]) (logid:8f9443c5) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:44,433 DEBUG [c.c.a.ApiServer] (qtp1215904751-317:[ctx-14f0eb9d, ctx-ed63be70]) (logid:8f9443c5) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:44,436 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-317:[ctx-14f0eb9d, ctx-ed63be70]) (logid:8f9443c5) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:44,436 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-317:[ctx-14f0eb9d, ctx-ed63be70]) (logid:8f9443c5) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:44,437 INFO [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-14f0eb9d, ctx-ed63be70]) (logid:8f9443c5) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:44,437 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-14f0eb9d, ctx-ed63be70]) (logid:8f9443c5) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:44,442 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-1:[ctx-16ddb573]) (logid:6b7e405a) HA health check task is running... 2025-02-19 10:36:46,319 DEBUG [c.c.u.s.SSHCmdHelper] (qtp1215904751-15:[ctx-be65162f, ctx-14aa0eee]) (logid:accee293) SSH command: cloudstack-setup-agent -m 192.168.1.20 -z 1 -p 1 -c 1 -g 7eb7429b-bf42-33cf-bc97-03f4d3d2c424 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm SSH command output:Starting to configure your system: Configure Host ... [OK] Configure Apparmor ... [OK] Configure Network ... Configure Network failed, Please check the /var/log/cloudstack/agent/setup.log for detail, due to:[Errno 2] No such file or directory: '/etc/network/interfaces' Try to restore your system: Restore Host ... [OK] Restore Apparmor ... [OK] 2025-02-19 10:36:48,444 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-2:[ctx-daf2b2a9]) (logid:ddf70284) HA health check task is running... 2025-02-19 10:36:49,430 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-f41588ae]) (logid:e3f422c7) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:49,430 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-f41588ae]) (logid:e3f422c7) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:49,437 DEBUG [c.c.a.ApiServer] (qtp1215904751-17:[ctx-f41588ae, ctx-d7b885c8]) (logid:e3f422c7) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:49,439 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-17:[ctx-f41588ae, ctx-d7b885c8]) (logid:e3f422c7) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:49,439 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-17:[ctx-f41588ae, ctx-d7b885c8]) (logid:e3f422c7) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:49,444 INFO [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-f41588ae, ctx-d7b885c8]) (logid:e3f422c7) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:49,444 DEBUG [c.c.a.ApiServlet] (qtp1215904751-17:[ctx-f41588ae, ctx-d7b885c8]) (logid:e3f422c7) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:52,447 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-4:[ctx-7dec2807]) (logid:e1c3e9b6) HA health check task is running... 2025-02-19 10:36:52,604 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-0bf99565]) (logid:5d7a290b) Enabled non-edge zones available for scan: 1 2025-02-19 10:36:52,605 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-0bf99565]) (logid:5d7a290b) Zone 1 has no host available which is enabled and in Up state 2025-02-19 10:36:52,605 DEBUG [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:[ctx-0bf99565]) (logid:5d7a290b) Zone [1] is not ready to launch secondary storage VM. 2025-02-19 10:36:52,621 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:[ctx-229f5ca4]) (logid:dc93f5c2) Skip capacity scan as there is no Primary Storage in 'Up' state 2025-02-19 10:36:54,432 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2f2c8b65]) (logid:99c53c74) ===START=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:54,432 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2f2c8b65]) (logid:99c53c74) Two factor authentication is already verified for the user 2, so skipping 2025-02-19 10:36:54,439 DEBUG [c.c.a.ApiServer] (qtp1215904751-317:[ctx-2f2c8b65, ctx-742549f9]) (logid:99c53c74) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"2b7da926-ed9c-11ef-838c-000c29e7ae90"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2025-02-19 10:36:54,440 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1215904751-317:[ctx-2f2c8b65, ctx-742549f9]) (logid:99c53c74) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2025-02-19 10:36:54,440 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1215904751-317:[ctx-2f2c8b65, ctx-742549f9]) (logid:99c53c74) API rate limiting is disabled. We will not use ApiRateLimitService. 2025-02-19 10:36:54,442 INFO [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2f2c8b65, ctx-742549f9]) (logid:99c53c74) (userId=2 accountId=2 sessionId=node0maw0vegqs5xy16hqy8rj7uvkq0) 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 200 {"readyforshutdownresponse":{"readyforshutdown":{"readyforshutdown":true,"shutdowntriggered":false,"pendingjobscount":0}}} 2025-02-19 10:36:54,442 DEBUG [c.c.a.ApiServlet] (qtp1215904751-317:[ctx-2f2c8b65, ctx-742549f9]) (logid:99c53c74) ===END=== 192.168.1.199 -- GET command=readyForShutdown&response=json&sessionkey=pPCweXbSXlsOq0_Ld76jsBBY9P8 2025-02-19 10:36:56,449 DEBUG [o.a.c.h.H.HAManagerBgPollTask] (BackgroundTaskPollManager-1:[ctx-2783fb5e]) (logid:b152a10e) HA health check task is running... 2025-02-19 10:36:58,962 DEBUG [c.c.s.S.DbCollector] (StatsCollector-1:[ctx-c8d34e45]) (logid:85b4b2b2) DbCollector is running... 2025-02-19 10:36:58,994 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-00844beb]) (logid:fed3cec7) Found 0 routers to update status. 2025-02-19 10:36:58,995 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-00844beb]) (logid:fed3cec7) Found 0 VPC's to update Redundant State. 2025-02-19 10:36:58,998 DEBUG [c.c.n.r.V.CheckRouterTask] (RouterStatusMonitor-1:[ctx-00844beb]) (logid:fed3cec7) Found 0 networks to update RvR status. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org