Can you try this:
1. Try the UI in a different browser or incognito mode to rule our UI-related caching issues 2. Have you upgraded all your management servers to 4.19.1.0? 3. And all your KVM hosts - are they all Up and in healthy states? Have you secured them all? For example, after upgrading your hosts you can ensure that libvirtd runs on TLS secured port 15914, or use this https://cloudstack.apache.org/api/apidocs-4.19/apis/provisionCertificate.html 4. As a workaround, you can set the auth strictness (ca.plugin.root.auth.strictness global setting via mgmt server UI) to false and try #3 5. Repeat your tests again by destroying your CPVM Regards. ________________________________ From: Biswajit Banerjee <[email protected]> Sent: Monday, August 5, 2024 11:23 To: [email protected] <[email protected]> Subject: Re: Upgrade to 4.19.1.0 from 4.15.1 - Issues Hi Experts , Please Help on the stated issues Thanks Biswajit On 8/2/24 18:58, Biswajit Banerjee wrote: > Yes , there are 23 KVM hosts , all has been upgraded . > > FYI we are still using Centos7.9 on all hosts . > > On 8/2/24 18:14, Wei ZHOU wrote: >> Have you upgraded all cloudstack-agent (if you use kvm) ? >> >> -Wei >> >> On Fri, Aug 2, 2024 at 2:33 PM Biswajit Banerjee >> <[email protected]> wrote: >>> After destroying the VM , it gets automatically recreated . I presume >>> that is what is expected . >>> >>> Let me Know if any thing else is required . >>> >>> Also about " 'readyForShutdown' either does not exist, is not >>> available for user. Unable to proceed " Please guide me >>> >>> On 8/2/24 17:59, Biswajit Banerjee wrote: >>>> Yes destroyed CPVM Many times but did not work . >>>> >>>> On 8/2/24 13:16, Wei ZHOU wrote: >>>>> Hi, >>>>> >>>>> Have you destroy/recreate the CPVM ? >>>>> >>>>> -Wei >>>>> >>>>> On Fri, Aug 2, 2024 at 12:55 AM Biswajit Banerjee >>>>> <[email protected]> wrote: >>>>>> Hello, >>>>>> >>>>>> We have Upgraded ACS 4.15.1 to 4.19.1.0 . Every thing are fine apart >>>>>> from 2 issues >>>>>> >>>>>> 1. We are getting repeated error on ACS webUI with admin users >>>>>> saying " >>>>>> The given command 'readyForShutdown' either does not exist, >>>>>> is not >>>>>> available for user. Unable to proceed. Please contact your >>>>>> administrator" every Second . how can we disable this repeated >>>>>> message . please Guide >>>>>> 2. Console proxy gives " Access is denied for the console session >>>>>> " and >>>>>> Following is the error in /var/log/cloud.log >>>>>> >>>>>> /A2024-08-01 22:38:48,121 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>> (Console-Proxy-Main:null) Setting reconnectMaxRetry=5 >>>>>> 2024-08-01 22:38:48,127 INFO >>>>>> [cloud.consoleproxy.ConsoleProxyBaseServerFactoryImpl] >>>>>> (Console-Proxy-Main:null) create HTTP server instance at port: 80 >>>>>> 2024-08-01 22:38:48,718 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>> (Console-Proxy-Main:null) Listening for HTTP CMDs on port 8001 >>>>>> 2024-08-01 22:39:29,274 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>> (Thread-12:null) Session null has already been used, cannot connect >>>>>> *2024-08-01 22:39:29,278 WARN [cloud.consoleproxy.ConsoleProxy] >>>>>> (Thread-12:null) External authenticator failed authentication >>>>>> request >>>>>> for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>> 2024-08-01 22:39:29,281 WARN >>>>>> [cloud.consoleproxy.ConsoleProxyAjaxHandler] (Thread-12:null) >>>>>> Failed to >>>>>> create viewer due to External authenticator failed request for vm >>>>>> 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid uDFk1uQZy9YBz5ZRSSB1SA >>>>>> com.cloud.consoleproxy.AuthenticationException: External >>>>>> authenticator >>>>>> failed request for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>> uDFk1uQZy9YBz5ZRSSB1SA* >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxy.authenticationExternally(ConsoleProxy.java:564) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxy.getAjaxVncViewer(ConsoleProxy.java:494) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.doHandle(ConsoleProxyAjaxHandler.java:142) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.handle(ConsoleProxyAjaxHandler.java:51) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:848) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:817) >>>>>> >>>>>> >>>>>> at java.base/java.lang.Thread.run(Thread.java:829) >>>>>> 2024-08-01 22:40:18,843 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>> (Thread-14:null) Session null has already been used, cannot connect >>>>>> 2024-08-01 22:40:18,861 WARN [cloud.consoleproxy.ConsoleProxy] >>>>>> (Thread-14:null) External authenticator failed authentication >>>>>> request >>>>>> for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>> 2024-08-01 22:40:18,862 WARN >>>>>> [cloud.consoleproxy.ConsoleProxyAjaxHandler] (Thread-14:null) >>>>>> Failed to >>>>>> create viewer due to External authenticator failed request for vm >>>>>> 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid uDFk1uQZy9YBz5ZRSSB1SA >>>>>> com.cloud.consoleproxy.AuthenticationException: External >>>>>> authenticator >>>>>> failed request for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxy.authenticationExternally(ConsoleProxy.java:564) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxy.getAjaxVncViewer(ConsoleProxy.java:494) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.doHandle(ConsoleProxyAjaxHandler.java:142) >>>>>> >>>>>> >>>>>> at >>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.handle(ConsoleProxyAjaxHandler.java:51) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:848) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>> >>>>>> >>>>>> at >>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:817) >>>>>> >>>>>> >>>>>> at java.base/java.lang.Thread.run(Thread.java:829) >>>>>> /Please guide us to resolve the issue . >>>>>> >>>>>> TIA >>>>>> >>>>>> Regards >>>>>> >>>>>> Biswajit
