On Mon, May 12, 2025 at 2:37 AM Zujian YU <[email protected]> wrote:
> Hello, > > > > We are running Gucamole solution on AWS environment, and it works for > years, recently we have issue to access it. The Gucamole EC2 server join > AWS managed AD, and the AWS managed AD setup on-way trust to On-Prem DC AD > which host the end user credentials. > > > > *Env*: > > OS: RHEL8.10 > > Gucamole: 1.5.5 > > Apache-tomcat: 9.0.100 > > > > *Configuration:* > > # cat /etc/guacamole/guacd.conf > > *[daemon]* > > *pid_file = /var/run/guacd.pid* > > *log_level = info* > > > > *[server]* > > *bind_host = 127.0.0.1* > > *bind_port = 4822* > > > > # cat /usr/share/tomcat/.guacamole/guacamole.properties > > > > *guacd-hostname: 127.0.0.1* > > *guacd-port: 4822* > > *auth-provider: > net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider* > > > Please note that the "auth-provider" property was removed a long time ago and has absolutely no effect on the configuration. > > > *Symptom:* > > 1. Some users login Gucamole portal with timeout error, and found > below logs: > > #cat /var/log/tomcat/catalina.out > > … > > *00:36:00.468 [NioProcessor-160] WARN* *o.a.d.l.c.api.LdapNetworkConnection > - null* > > *org.apache.mina.core.write.WriteToClosedSessionException: null* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)* > > *at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)* > > *at java.base/java.lang.Thread.run(Thread.java:829)* > > *00:36:01.011 [http-nio-8080-exec-21] INFO* > *o.a.g.a.l.AuthenticationProviderService > - User "userid-***" was successfully authenticated by LDAP server > "onprem-ad-***.com".* > > *00:36:01.012 [http-nio-8080-exec-21] INFO* > *o.a.g.r.auth.AuthenticationService > - User "userid-***" successfully authenticated from [10.139.12.175, > 10.73.192.28, 127.0.0.1].* > > *00:36:11.072 [NioProcessor-162] WARN* *o.a.d.l.c.api.LdapNetworkConnection > - null* > > *org.apache.mina.core.write.WriteToClosedSessionException: null* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)* > > *at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)* > > *at java.base/java.lang.Thread.run(Thread.java:829)* > > *00:36:11.606 [http-nio-8080-exec-26] INFO* > *o.a.g.a.l.AuthenticationProviderService > - User "userid-***" was successfully authenticated by LDAP server > "onprem-ad-***.com".* > > *00:36:11.606 [http-nio-8080-exec-26] INFO* > *o.a.g.r.auth.AuthenticationService > - User "userid-***" successfully authenticated from [10.139.12.175, > 10.73.192.28, 127.0.0.1].* > > *00:36:13.754 [NioProcessor-163] WARN* *o.a.d.l.c.api.LdapNetworkConnection > - null* > > *org.apache.mina.core.write.WriteToClosedSessionException: null* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)* > > *at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)* > > *at java.base/java.lang.Thread.run(Thread.java:829)* > > *00:36:16.119 [http-nio-8080-exec-26] INFO* > *o.a.g.tunnel.TunnelRequestService > - User "userid-***" connected to connection "1787".* > > *00:36:16.119 [http-nio-8080-exec-26] INFO* > *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet > - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.* > > *00:36:26.010 [http-nio-8080-exec-27] INFO* > *o.a.g.tunnel.TunnelRequestService > - User "userid-***" disconnected from connection "1787". Duration: 9891 > milliseconds* > > *00:36:32.609 [http-nio-8080-exec-19] INFO* > *o.a.g.tunnel.TunnelRequestService > - User "userid-***" connected to connection "1787".* > > *00:36:32.609 [http-nio-8080-exec-19] INFO* > *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet > - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.* > > *00:36:50.343 [http-nio-8080-exec-2] INFO* *o.a.g.tunnel.TunnelRequestService > - User "userid-***" disconnected from connection "1787". Duration: 17734 > milliseconds* > > *00:36:50.555 [http-nio-8080-exec-23] WARN* > *o.a.g.s.GuacamoleHTTPTunnelServlet > - HTTP tunnel request rejected: No such tunnel.* > > *00:37:33.522 [http-nio-8080-exec-27] INFO* > *o.a.g.tunnel.TunnelRequestService > - User "userid-***" connected to connection "1787".* > > *00:37:33.522 [http-nio-8080-exec-27] INFO* > *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet > - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.* > > *00:37:49.866 [http-nio-8080-exec-2] INFO* *o.a.g.tunnel.TunnelRequestService > - User "userid-***" disconnected from connection "1787". Duration: 16344 > milliseconds* > > *00:37:50.071 [http-nio-8080-exec-21] WARN* > *o.a.g.s.GuacamoleHTTPTunnelServlet > - HTTP tunnel request rejected: No such tunnel.* > > > > … > > > > > > 2. I can login Gucamole portal but when click the AWS EC2 server to > ssh, error log show: > > > > *1:28:06.312 [NioProcessor-183] WARN* *o.a.d.l.c.api.LdapNetworkConnection > - null* > > *org.apache.mina.core.write.WriteToClosedSessionException: null* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)* > > *at > org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)* > > *at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)* > > *at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)* > > *at java.base/java.lang.Thread.run(Thread.java:829)* > > *01:29:59.444 [http-nio-8080-exec-23] INFO* > *o.a.g.tunnel.TunnelRequestService > - User “***” connected to connection "1874".* > > *01:29:59.444 [http-nio-8080-exec-23] INFO* > *o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet > - Using HTTP tunnel (not WebSocket). Performance may be sub-optimal.* > > *01:31:14.621 [http-nio-8080-exec-5] INFO* *o.a.g.tunnel.TunnelRequestService > - User “***” disconnected from connection "1874". Duration: 75177 > milliseconds* > > *01:31:14.635 [http-nio-8080-exec-5] ERROR > o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request failed: Connection > to guacd timed out.* > > *01:31:14.808 [http-nio-8080-exec-21] WARN* > *o.a.g.s.GuacamoleHTTPTunnelServlet > - HTTP tunnel request rejected: No such tunnel.* > > *01:31:14.814 [http-nio-8080-exec-4] WARN* > *o.a.g.s.GuacamoleHTTPTunnelServlet > - HTTP tunnel request rejected: No such tunnel.* > > > > *Verfication:* > > - AWS subnet NACLs / Guacamole EC2 security group rules are all good > to whitelist required traffic. > - The Gucamole EC2 join domain successfully > > *[...]* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending > normalization specified. Defaulting to preserving the format of all line > endings.* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User > "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection > "$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will > be saved to "/opt/guacamole/recording/screen/992382480584 - > mx-asg-db-connect-j13555-j13555-ap-southeast-3 - > i-0b77f582013cbc860//202505>* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"* > > *May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap > "en-us-qwerty"* > > *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused > connection: Disconnected.* > > *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User > "@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)* > > *May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection > "$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected* > > *May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection > "$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.* > > > > Any suggestion is appreciated. > > > I think some more network-level troubleshooting is going to be required - all of the messages you posted and symptoms you describe appear to be network-related. In the immediate message above, the reason for the failure is given: RDP server closed/refused connection: Disconnected. This indicates that, for one reason or another, Guacamole could not establish the connection to the RDP server. Could be a VPC-level issue (routing, ACL, Security Groups, etc.), or it could be a configuration issue (wrong security level, for example). Combine this with the LDAP issues you mention earlier, and I would definitely say something network-related is going on - Guacamole is having trouble consistently connecting to or maintaining a connection to the LDAP server. Aside from network issues, you might check resources utilization - depending on how many users you have logging in concurrently, it could be that network connections are failing or getting dropped because the system running Guacamole does not have the RAM or CPU to handle the number of connections. Also, since you're running RHEL, you might want to make sure that SELinux is not interfering with the connections. I don't recommend disabling SELinux entirely, but you can use audit2why to examine the /var/log/audit/audit.log file and see if SELinux is dropping or preventing any connections. -Nick >
