Hi Mike, Nick, Running out of ideas now, at least until the Firewall vendor responds to my support case.
I have set the enable-websocket: false and also now changed Tomcat to SSL support, as shown in this syslog entry: “Sep 27 15:50:33 psmguc01 tomcat9[142913]: 15:50:33.634 [https-openssl-nio-8443-e xec-15] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal." Still no joy, am in the same boat. While the connection is not working, I see this in the syslog once I close it manually: “Sep 27 15:50:34 psmguc01 guacd[143114]: Received nop instruction Sep 27 15:50:39 psmguc01 guacd[143114]: message repeated 11 times: [ Received nop instruction] Sep 27 15:50:48 psmguc01 guacd[143104]: User is not responding." The syslog shows this as unexpected message, which I doubt is causing the issue: “Sep 27 15:50:33 psmguc01 guacd[143114]: "HOME" environment variable was unset and has been automatically set to "/root" I have a the Catalina log entry from 2 connection attempts, and even though WebSocket is disabled, it seems the first connection attempt still tries to use it. To close the session, I have to backpage, then go to the sessions page and kill it, before starting the other connection attempt. [2021-09-27 16:15:34] [info] 16:15:34.842 [https-openssl-nio-8443-exec-17] INFO o.a.g.r.auth.AuthenticationService - User "guactest" successfully authenticated from 192.168.106.1. [2021-09-27 16:15:40] [info] 16:15:40.632 [https-openssl-nio-8443-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" connected to connection "12". [2021-09-27 16:15:40] [info] 16:15:40.641 [https-openssl-nio-8443-exec-19] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" disconnected from connection "12". Duration: 9 milliseconds [2021-09-27 16:15:40] [info] Exception in thread "Thread-10" java.lang.IllegalStateException: Message will not be sent because the WebSocket session has been closed [2021-09-27 16:15:40] [info] #011at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:430) [2021-09-27 16:15:40] [info] #011at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:309) [2021-09-27 16:15:40] [info] #011at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:250) [2021-09-27 16:15:40] [info] #011at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemoteEndpointImplBase.java:191) [2021-09-27 16:15:40] [info] #011at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:37) [2021-09-27 16:15:40] [info] #011at org.apache.guacamole.websocket.GuacamoleWebSocketTunnelEndpoint.sendInstruction(GuacamoleWebSocketTunnelEndpoint.java:152) [2021-09-27 16:15:40] [info] #011at org.apache.guacamole.websocket.GuacamoleWebSocketTunnelEndpoint.sendInstruction(GuacamoleWebSocketTunnelEndpoint.java:172) [2021-09-27 16:15:40] [info] #011at org.apache.guacamole.websocket.GuacamoleWebSocketTunnelEndpoint.access$100(GuacamoleWebSocketTunnelEndpoint.java:53) [2021-09-27 16:15:40] [info] #011at org.apache.guacamole.websocket.GuacamoleWebSocketTunnelEndpoint$2.run(GuacamoleWebSocketTunnelEndpoint.java:238) [2021-09-27 16:15:40] [info] 16:15:40.705 [https-openssl-nio-8443-exec-17] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" connected to connection "12". [2021-09-27 16:15:40] [info] 16:15:40.705 [https-openssl-nio-8443-exec-17] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal. [2021-09-27 16:16:14] [info] 16:16:14.893 [https-openssl-nio-8443-exec-11] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" disconnected from connection "12". Duration: 34188 milliseconds [2021-09-27 16:16:14] [info] 16:16:14.922 [https-openssl-nio-8443-exec-8] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel. [2021-09-27 16:16:14] [info] 16:16:14.924 [https-openssl-nio-8443-exec-2] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel. [2021-09-27 16:16:31] [info] 16:16:31.592 [https-openssl-nio-8443-exec-15] ERROR o.a.g.w.GuacamoleWebSocketTunnelEndpoint - Creation of WebSocket tunnel to guacd failed: Cannot connect. Connection already in use by this user. [2021-09-27 16:16:31] [info] 16:16:31.655 [https-openssl-nio-8443-exec-5] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" connected to connection "9". [2021-09-27 16:16:31] [info] 16:16:31.655 [https-openssl-nio-8443-exec-5] INFO o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal. [2021-09-27 16:17:04] [info] 16:17:04.837 [https-openssl-nio-8443-exec-8] INFO o.a.g.tunnel.TunnelRequestService - User "guactest" disconnected from connection "9". Duration: 33182 milliseconds [2021-09-27 16:17:04] [info] 16:17:04.868 [https-openssl-nio-8443-exec-7] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel. [2021-09-27 16:17:04] [info] 16:17:04.871 [https-openssl-nio-8443-exec-12] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: No such tunnel. Regards, Craig From: Stratton, Craig <[email protected]> Sent: 27 September 2021 09:10 To: [email protected] Subject: RE: Exhausted simultaneous connection error This message originated from outside your organization ________________________________ Hi Mike, Thanks for the response. Having thought more on it over the weekend, I think that is the right area. The server is in a DMZ, and if I connect directly from within the network (although still going through firewall) it works correctly. Where it is intermittent (or no longer working, as now) I am trying to use it from outside the firewall, using an SSL Proxy Portal service on the firewall itself. So I think the firewall is indeed ignoring the WebSocket connection coming back, and the failback to HTTP was happening faster on the times it did work. Is there a way to disable the WebSocket support, or reduce the fallback timer, while I identify or resolve the issue on the firewall? I read an alternative would be to convert the Tomcat instance to SSL, if it was a buffering issue, but I am not sure it would fix this if it is a WebSocket issue? Will test again today and check the logs for confirmation, and discuss with firewall vendor for some specific SSL Portal info and logging. Thanks, Craig From: Mike Jumper <[email protected]<mailto:[email protected]>> Sent: 24 September 2021 18:46 To: [email protected]<mailto:[email protected]> Subject: Re: Exhausted simultaneous connection error This message originated from outside your organization ________________________________ I believe there are cases where this error can appear due to WebSocket being inadvertently blocked by a network device or proxy. If the WebSocket connection attempt fails due to certain kinds of interference, the browser will abruptly abort the connection attempt and server-side resources for that connection will not be released by the time the client retries using HTTP. Do you see any warnings in the logs regarding WebSocket and the HTTP fallback? Anything on the network that might be interfering? - Mike On Fri, Sep 24, 2021, 08:00 Stratton, Craig <[email protected]<mailto:[email protected]>> wrote: Hi Nick, Guacd version 1.3.0 running native on Ubuntu 20.04 Apologies, I had read and understood that guacd should not be the problem and did not need restarting, but wrote that anyway for some reason. I had recently restarted it to change the loglevel. Client has been complied with Postgres, RADIUS and LDAP authentication, although could not get RADIUS to work and is disabled. User is authenticated against LDAP, and database Groups match defined LDAP groups, so no users defined in local database, they see database defined connections based on LDAP group membership. This all works as expected. Thank you, Craig From: Nick Couchman <[email protected]<mailto:[email protected]>> Sent: 24 September 2021 14:42 To: [email protected]<mailto:[email protected]> Subject: Re: Exhausted simultaneous connection error This message originated from outside your organization ________________________________ On Fri, Sep 24, 2021 at 7:48 AM Stratton, Craig <[email protected]<mailto:[email protected]>> wrote: Hi, I am continually running into this error and cannot seem to resolve it. “The Guacamole server is denying access to this connection because you have exhausted the limit for simultaneous connection use by an individual user. Please close one or more connections and try again.” There are no connections listed for the user when I look to close them. I have some connections set with default blank number of connections per user, some with 1 some with 10, but it happens on all of them. I can connect, disconnect, reconnect fine after creating a new connection, then if I try again the following day I get that error, even after closing properly. I have not set any of the guacamole.properties<http://guacamole.properties> file entries to override any defaults on number of connections, as the way I read the manual, there are no limits by default. If I stop and restart guacd and tomcat, it makes no difference and still cannot connect, it will just randomly start working again after some undetermined timeout? Just to note, here, guacd is not related to this issue, as the connection tracking, including simultaneous connections, is done by Tomcat/Guacamole Client. I say that only to note that restarting guacd isn't going to do anything for this. Restarting Tomcat should clear things out, but you shouldn't need to mess with guacd. That said, guacd logs may help you to determine if an unexpected connection is coming through, so might not be a bad idea to pay attention to those. What version of Guacamole are you running? What configuration - Docker or native, MySQL, Postgres, etc.? -NIck Public Sector Partnership Services Limited (PSPS) is a Local Authority Trading Company, wholly owned by East Lindsey District Council, South Holland District Council and Boston Borough Council in Lincolnshire. PSPS delivers services to and on behalf of the three District Councils. Registered Company details: Public Sector Partnership Services Limited, 2 New Bailey, 6 Stanley Street, Salford, Greater Manchester M3 5GS Registered in England, Number – 07289357 Confidentiality: This e-mail and its attachments are intended for the above named only and may contain confidential and privileged information. If you are not the intended recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please notify the sender. The views expressed in this message are my own, and any negotiations by email are subject to formal contract. Any correspondence with the sender will be subject to automatic monitoring for inappropriate content. Your information will be processed in accordance with the law, in particular current Data Protection legislation. If you have contacted Public Sector Partnership Services for a service then your personal data will be processed in order to provide that service or answer your enquiry. For full details of our Privacy Policy and your rights please go to our website at https://www.pspsl.co.uk/privacy<https://www.pspsl.co.uk/privacy>. The information that you provide will only be used for Company purposes unless there is a legal authority to do otherwise. The contents of e-mails may have to be disclosed to a request under the Data Protection Act and the Freedom of Information Act 2000. Public Sector Partnership Services Limited (PSPS) is a Local Authority Trading Company, wholly owned by East Lindsey District Council, South Holland District Council and Boston Borough Council in Lincolnshire. PSPS delivers services to and on behalf of the three District Councils. Registered Company details: Public Sector Partnership Services Limited, 2 New Bailey, 6 Stanley Street, Salford, Greater Manchester M3 5GS Registered in England, Number – 07289357 Confidentiality: This e-mail and its attachments are intended for the above named only and may contain confidential and privileged information. If you are not the intended recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please notify the sender. The views expressed in this message are my own, and any negotiations by email are subject to formal contract. Any correspondence with the sender will be subject to automatic monitoring for inappropriate content. Your information will be processed in accordance with the law, in particular current Data Protection legislation. If you have contacted Public Sector Partnership Services for a service then your personal data will be processed in order to provide that service or answer your enquiry. For full details of our Privacy Policy and your rights please go to our website at https://www.pspsl.co.uk/privacy<https://www.pspsl.co.uk/privacy>. The information that you provide will only be used for Company purposes unless there is a legal authority to do otherwise. The contents of e-mails may have to be disclosed to a request under the Data Protection Act and the Freedom of Information Act 2000. Public Sector Partnership Services Limited (PSPS) is a Local Authority Trading Company, wholly owned by East Lindsey District Council, South Holland District Council and Boston Borough Council in Lincolnshire. PSPS delivers services to and on behalf of the three District Councils. Registered Company details: Public Sector Partnership Services Limited, 2 New Bailey, 6 Stanley Street, Salford, Greater Manchester M3 5GS Registered in England, Number – 07289357 Confidentiality: This e-mail and its attachments are intended for the above named only and may contain confidential and privileged information. If you are not the intended recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please notify the sender. The views expressed in this message are my own, and any negotiations by email are subject to formal contract. Any correspondence with the sender will be subject to automatic monitoring for inappropriate content. Your information will be processed in accordance with the law, in particular current Data Protection legislation. If you have contacted Public Sector Partnership Services for a service then your personal data will be processed in order to provide that service or answer your enquiry. For full details of our Privacy Policy and your rights please go to our website at https://www.pspsl.co.uk/privacy. The information that you provide will only be used for Company purposes unless there is a legal authority to do otherwise. The contents of e-mails may have to be disclosed to a request under the Data Protection Act and the Freedom of Information Act 2000.
