All, If you're facing this issues, chances are you are using an old/deprecated static role checker. Please see if enabling the dynamic checker and restarting the management server fixes your issue - https://github.com/apache/cloudstack/issues/9491#issuecomment-2272758422
Regards. ________________________________ From: Janis Viklis | Files.fm <[email protected]> Sent: Tuesday, August 6, 2024 18:58 To: [email protected] <[email protected]> Subject: Re: Upgrade to 4.19.1.0 from 4.15.1 - Issues +1. Same issues, multiple/different messages depending on user type: Root admin: The given command 'readyForShutdown' either does not exist, is not available for user. Unable to proceed. Please contact your administrator. User: The given command 'listAnnotations' either does not exist, is not available for user. Unable to proceed. Please contact your administrator. https://cloudstack_host/client/api/?command=readyForShutdown&response=json { "readyforshutdownresponse": { "uuidList": [], "errorcode": 401, "cserrorcode": 9999, "errortext": "The given command 'readyForShutdown' either does not exist, is not available for user." } } 2024-08-06 16:24:30,351 DEBUG [c.c.a.ApiServlet] (qtp1386883398-1735:ctx-929d21c8) (logid:c95b3248) ===START=== 10.10.10.31 -- GET command=readyForShutdown&response=json 2024-08-06 16:24:30,351 DEBUG [c.c.a.ApiServlet] (qtp1386883398-1735:ctx-929d21c8) (logid:c95b3248) Two factor authentication is already verified for the user 2, so skipping 2024-08-06 16:24:30,375 DEBUG [c.c.a.ApiServer] (qtp1386883398-1735:ctx-929d21c8 ctx-0017b621) (logid:c95b3248) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"f0e7fdf6-48f1-11e6-a9f1-00163e44393e"}]' is allowed to perform API calls: 0.0.0.0/0 2024-08-06 16:24:30,391 DEBUG [c.c.a.ApiServlet] (qtp1386883398-1735:ctx-929d21c8 ctx-0017b621) (logid:c95b3248) ===END=== 10.10.10.31 -- GET command=readyForShutdown&response=json Janis On 2024-08-06 10:25, Biswajit Banerjee wrote: > Github issue raise at https://github.com/apache/cloudstack/issues/9491 > > On 8/6/24 10:54, Biswajit Banerjee wrote: >> Thanks Rohit for quick response >> >> We have created account and assigned them to root admin roles . does >> it means custom root admin role ? >> >> We will raise the case at github . >> >> On 8/5/24 16:20, Rohit Yadav wrote: >>> Hi Biswajit - are you using custom Root Admin roles? >>> >>> The CloudStack safe shutdown feature added this API >>> (https://github.com/apache/cloudstack/pull/6755) and you may need to >>> check and allow this API for your root admin roles if they already >>> don't have this API allowed. However, I sense skimming quickly the >>> feature doesn't seems to allow a way to disable it - perhaps you can >>> review for your use-cases and log an issue here - >>> https://github.com/apache/cloudstack/issues >>> >>> >>> Regards. >>> >>> >>> >>> ________________________________ >>> From: Biswajit Banerjee <[email protected]> >>> Sent: Monday, August 5, 2024 15:18 >>> To: [email protected] <[email protected]> >>> Subject: Re: Upgrade to 4.19.1.0 from 4.15.1 - Issues >>> >>> Thanks Rohit >>> >>> Our console access has been sorted out by enabling novnc console via >>> global config . >>> >>> Can You Please help us with >>> >>> We are getting repeated error on ACS webUI with admin users saying "The >>> given command '*readyForShutdown'* either does not exist, is not >>> available for user. Unable to proceed. Please contact your >>> administrator" every Second . how can we disable this repeated >>> message . >>> >>> Thanks >>> >>> Biswajit >>> >>> On 8/5/24 11:31, Rohit Yadav wrote: >>>> Can you try this: >>>> >>>> >>>> 1. >>>> Try the UI in a different browser or incognito mode to rule our >>>> UI-related caching issues >>>> 2. >>>> Have you upgraded all your management servers to 4.19.1.0? >>>> 3. >>>> And all your KVM hosts - are they all Up and in healthy states? >>>> Have you secured them all? For example, after upgrading your hosts >>>> you can ensure that libvirtd runs on TLS secured port 15914, or use >>>> thishttps://cloudstack.apache.org/api/apidocs-4.19/apis/provisionCertificate.html >>>> 4. >>>> As a workaround, you can set the auth strictness >>>> (ca.plugin.root.auth.strictness global setting via mgmt server UI) >>>> to false and try #3 >>>> 5. >>>> Repeat your tests again by destroying your CPVM >>>> >>>> >>>> >>>> Regards. >>>> >>>> >>>> >>>> >>>> ________________________________ >>>> From: Biswajit Banerjee<[email protected]> >>>> Sent: Monday, August 5, 2024 11:23 >>>> To:[email protected] <[email protected]> >>>> Subject: Re: Upgrade to 4.19.1.0 from 4.15.1 - Issues >>>> >>>> Hi Experts , >>>> >>>> Please Help on the stated issues >>>> >>>> Thanks >>>> >>>> Biswajit >>>> >>>> On 8/2/24 18:58, Biswajit Banerjee wrote: >>>>> Yes , there are 23 KVM hosts , all has been upgraded . >>>>> >>>>> FYI we are still using Centos7.9 on all hosts . >>>>> >>>>> On 8/2/24 18:14, Wei ZHOU wrote: >>>>>> Have you upgraded all cloudstack-agent (if you use kvm) ? >>>>>> >>>>>> -Wei >>>>>> >>>>>> On Fri, Aug 2, 2024 at 2:33 PM Biswajit Banerjee >>>>>> <[email protected]> wrote: >>>>>>> After destroying the VM , it gets automatically recreated . I >>>>>>> presume >>>>>>> that is what is expected . >>>>>>> >>>>>>> Let me Know if any thing else is required . >>>>>>> >>>>>>> Also about " 'readyForShutdown' either does not exist, is not >>>>>>> available for user. Unable to proceed " Please guide me >>>>>>> >>>>>>> On 8/2/24 17:59, Biswajit Banerjee wrote: >>>>>>>> Yes destroyed CPVM Many times but did not work . >>>>>>>> >>>>>>>> On 8/2/24 13:16, Wei ZHOU wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Have you destroy/recreate the CPVM ? >>>>>>>>> >>>>>>>>> -Wei >>>>>>>>> >>>>>>>>> On Fri, Aug 2, 2024 at 12:55 AM Biswajit Banerjee >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> We have Upgraded ACS 4.15.1 to 4.19.1.0 . Every thing are >>>>>>>>>> fine apart >>>>>>>>>> from 2 issues >>>>>>>>>> >>>>>>>>>> 1. We are getting repeated error on ACS webUI with admin >>>>>>>>>> users >>>>>>>>>> saying " >>>>>>>>>> The given command 'readyForShutdown' either does not >>>>>>>>>> exist, >>>>>>>>>> is not >>>>>>>>>> available for user. Unable to proceed. Please contact >>>>>>>>>> your >>>>>>>>>> administrator" every Second . how can we disable this >>>>>>>>>> repeated >>>>>>>>>> message . please Guide >>>>>>>>>> 2. Console proxy gives " Access is denied for the >>>>>>>>>> console session >>>>>>>>>> " and >>>>>>>>>> Following is the error in /var/log/cloud.log >>>>>>>>>> >>>>>>>>>> /A2024-08-01 22:38:48,121 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Console-Proxy-Main:null) Setting reconnectMaxRetry=5 >>>>>>>>>> 2024-08-01 22:38:48,127 INFO >>>>>>>>>> [cloud.consoleproxy.ConsoleProxyBaseServerFactoryImpl] >>>>>>>>>> (Console-Proxy-Main:null) create HTTP server instance at >>>>>>>>>> port: 80 >>>>>>>>>> 2024-08-01 22:38:48,718 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Console-Proxy-Main:null) Listening for HTTP CMDs on port 8001 >>>>>>>>>> 2024-08-01 22:39:29,274 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Thread-12:null) Session null has already been used, cannot >>>>>>>>>> connect >>>>>>>>>> *2024-08-01 22:39:29,278 WARN [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Thread-12:null) External authenticator failed authentication >>>>>>>>>> request >>>>>>>>>> for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>>>>>> 2024-08-01 22:39:29,281 WARN >>>>>>>>>> [cloud.consoleproxy.ConsoleProxyAjaxHandler] (Thread-12:null) >>>>>>>>>> Failed to >>>>>>>>>> create viewer due to External authenticator failed request >>>>>>>>>> for vm >>>>>>>>>> 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>>>>>> com.cloud.consoleproxy.AuthenticationException: External >>>>>>>>>> authenticator >>>>>>>>>> failed request for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 >>>>>>>>>> with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA* >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxy.authenticationExternally(ConsoleProxy.java:564) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxy.getAjaxVncViewer(ConsoleProxy.java:494) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.doHandle(ConsoleProxyAjaxHandler.java:142) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.handle(ConsoleProxyAjaxHandler.java:51) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:848) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:817) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at java.base/java.lang.Thread.run(Thread.java:829) >>>>>>>>>> 2024-08-01 22:40:18,843 INFO [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Thread-14:null) Session null has already been used, cannot >>>>>>>>>> connect >>>>>>>>>> 2024-08-01 22:40:18,861 WARN [cloud.consoleproxy.ConsoleProxy] >>>>>>>>>> (Thread-14:null) External authenticator failed authentication >>>>>>>>>> request >>>>>>>>>> for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>>>>>> 2024-08-01 22:40:18,862 WARN >>>>>>>>>> [cloud.consoleproxy.ConsoleProxyAjaxHandler] (Thread-14:null) >>>>>>>>>> Failed to >>>>>>>>>> create viewer due to External authenticator failed request >>>>>>>>>> for vm >>>>>>>>>> 3cdf6590-ffa2-40e8-966c-63cc42534c26 with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>>>>>> com.cloud.consoleproxy.AuthenticationException: External >>>>>>>>>> authenticator >>>>>>>>>> failed request for vm 3cdf6590-ffa2-40e8-966c-63cc42534c26 >>>>>>>>>> with sid >>>>>>>>>> uDFk1uQZy9YBz5ZRSSB1SA >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxy.authenticationExternally(ConsoleProxy.java:564) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxy.getAjaxVncViewer(ConsoleProxy.java:494) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.doHandle(ConsoleProxyAjaxHandler.java:142) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.cloud.consoleproxy.ConsoleProxyAjaxHandler.handle(ConsoleProxyAjaxHandler.java:51) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:848) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:817) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> at java.base/java.lang.Thread.run(Thread.java:829) >>>>>>>>>> /Please guide us to resolve the issue . >>>>>>>>>> >>>>>>>>>> TIA >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> >>>>>>>>>> Biswajit >
