Hello,
We are running Gucamole solution on AWS environment, and it works for years,
recently we have issue to access it. The Gucamole EC2 server join AWS managed
AD, and the AWS managed AD setup on-way trust to On-Prem DC AD which host the
end user credentials.
Env:
OS: RHEL8.10
Gucamole: 1.5.5
Apache-tomcat: 9.0.100
Configuration:
# cat /etc/guacamole/guacd.conf
[daemon]
pid_file = /var/run/guacd.pid
log_level = info
[server]
bind_host = 127.0.0.1
bind_port = 4822
# cat /usr/share/tomcat/.guacamole/guacamole.properties
guacd-hostname: 127.0.0.1
guacd-port: 4822
auth-provider:
net.sourceforge.guacamole.net.auth.mysql.MySQLAuthenticationProvider
mysql-hostname: rds-url
mysql-port: 3306
mysql-database: dbname
mysql-username: username
mysql-password: passwd
mysql-default-max-connections-per-user: 0
mysql-default-max-group-connections-per-user: 0
# LDAP properties
ldap-hostname: onprem-ad-***.com
ldap-port: 636
ldap-encryption-method: ssl
ldap-search-bind-dn: CN=**,OU=**,OU=**,OU=**,DC=**,DC=**,DC=com
ldap-search-bind-password: ********
ldap-user-base-dn: DC=**,DC=**,DC=com
ldap-username-attribute: sAMAccountName
ldap-user-search-filter:
(memberOf=CN=**,OU=**,OU=**,OU=**,OU=**,DC=**,DC=**,DC=com)
# TOTP properties
totp-issuer AWS 3.0 (MAIN) - **
totp-mode sha1
Symptom:
1. Some users login Gucamole portal with timeout error, and found below logs:
#cat /var/log/tomcat/catalina.out
…
00:36:00.468 [NioProcessor-160] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:01.011 [http-nio-8080-exec-21] INFO
o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully
authenticated by LDAP server "onprem-ad-***.com".
00:36:01.012 [http-nio-8080-exec-21] INFO o.a.g.r.auth.AuthenticationService -
User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28,
127.0.0.1].
00:36:11.072 [NioProcessor-162] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:11.606 [http-nio-8080-exec-26] INFO
o.a.g.a.l.AuthenticationProviderService - User "userid-***" was successfully
authenticated by LDAP server "onprem-ad-***.com".
00:36:11.606 [http-nio-8080-exec-26] INFO o.a.g.r.auth.AuthenticationService -
User "userid-***" successfully authenticated from [10.139.12.175, 10.73.192.28,
127.0.0.1].
00:36:13.754 [NioProcessor-163] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
00:36:16.119 [http-nio-8080-exec-26] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" connected to connection "1787".
00:36:16.119 [http-nio-8080-exec-26] INFO
o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
WebSocket). Performance may be sub-optimal.
00:36:26.010 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" disconnected from connection "1787". Duration: 9891
milliseconds
00:36:32.609 [http-nio-8080-exec-19] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" connected to connection "1787".
00:36:32.609 [http-nio-8080-exec-19] INFO
o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
WebSocket). Performance may be sub-optimal.
00:36:50.343 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" disconnected from connection "1787". Duration: 17734
milliseconds
00:36:50.555 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet -
HTTP tunnel request rejected: No such tunnel.
00:37:33.522 [http-nio-8080-exec-27] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" connected to connection "1787".
00:37:33.522 [http-nio-8080-exec-27] INFO
o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
WebSocket). Performance may be sub-optimal.
00:37:49.866 [http-nio-8080-exec-2] INFO o.a.g.tunnel.TunnelRequestService -
User "userid-***" disconnected from connection "1787". Duration: 16344
milliseconds
00:37:50.071 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet -
HTTP tunnel request rejected: No such tunnel.
…
1. I can login Gucamole portal but when click the AWS EC2 server to ssh,
error log show:
1:28:06.312 [NioProcessor-183] WARN o.a.d.l.c.api.LdapNetworkConnection - null
org.apache.mina.core.write.WriteToClosedSessionException: null
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:1192)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeNow(AbstractPollingIoProcessor.java:1153)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.removeSessions(AbstractPollingIoProcessor.java:864)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:694)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
01:29:59.444 [http-nio-8080-exec-23] INFO o.a.g.tunnel.TunnelRequestService -
User “***” connected to connection "1874".
01:29:59.444 [http-nio-8080-exec-23] INFO
o.a.g.t.h.RestrictedGuacamoleHTTPTunnelServlet - Using HTTP tunnel (not
WebSocket). Performance may be sub-optimal.
01:31:14.621 [http-nio-8080-exec-5] INFO o.a.g.tunnel.TunnelRequestService -
User “***” disconnected from connection "1874". Duration: 75177 milliseconds
01:31:14.635 [http-nio-8080-exec-5] ERROR o.a.g.s.GuacamoleHTTPTunnelServlet -
HTTP tunnel request failed: Connection to guacd timed out.
01:31:14.808 [http-nio-8080-exec-21] WARN o.a.g.s.GuacamoleHTTPTunnelServlet -
HTTP tunnel request rejected: No such tunnel.
01:31:14.814 [http-nio-8080-exec-4] WARN o.a.g.s.GuacamoleHTTPTunnelServlet -
HTTP tunnel request rejected: No such tunnel.
Verfication:
* AWS subnet NACLs / Guacamole EC2 security group rules are all good to
whitelist required traffic.
* The Gucamole EC2 join domain successfully
# net ads info
LDAP server: 10.*.*.*
LDAP server name: * (AD server of AWS managed AD)
Realm: *.com (the domain of AWS managed AD)
Bind Path: dc=*,dc=*,dc=*,dc=*,dc=*,dc=COM
LDAP port: 389
Server time: Mon, 12 May 2025 06:17:04 UTC
KDC server: 10.*.*.* (same as LDAP server)
Server time offset: 0
Last machine account password change: Mon, 05 May 2025 06:03:02 UTC
* Tomcat and guacd status ( also restarted it but not works)
# systemctl status tomcat guacd
● tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/etc/systemd/system/tomcat.service; enabled; vendor preset:
disabled)
Active: active (running) since Mon 2025-05-12 03:31:12 UTC; 2h 58min ago
Process: 1780229 ExecStop=/usr/share/tomcat/bin/shutdown.sh (code=exited,
status=0/SUCCESS)
Process: 1780349 ExecStart=/usr/share/tomcat/bin/startup.sh (code=exited,
status=0/SUCCESS)
Main PID: 1780359 (java)
Tasks: 40 (limit: 99182)
Memory: 772.6M
CGroup: /system.slice/tomcat.service
└─1780359 /usr/lib/jvm/jre-openjdk/bin/java
-Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.awt.head>
May 12 03:31:12 ac152d4d39a4432 systemd[1]: Starting Apache Tomcat Web
Application Container...
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Existing PID file found
during start.
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Removing/clearing stale
PID file.
May 12 03:31:12 ac152d4d39a4432 startup.sh[1780349]: Tomcat started.
May 12 03:31:12 ac152d4d39a4432 systemd[1]: Started Apache Tomcat Web
Application Container.
● guacd.service - LSB: Guacamole proxy daemon
Loaded: loaded (/etc/rc.d/init.d/guacd; generated)
Active: active (running) since Mon 2025-05-12 03:31:18 UTC; 2h 57min ago
Docs: man:systemd-sysv-generator(8)
Process: 1780442 ExecStop=/etc/rc.d/init.d/guacd stop (code=exited,
status=0/SUCCESS)
Process: 1780445 ExecStart=/etc/rc.d/init.d/guacd start (code=exited,
status=0/SUCCESS)
Tasks: 1 (limit: 99182)
Memory: 10.5M
CGroup: /system.slice/guacd.service
└─1780450 /usr/local/sbin/guacd -p /var/run/guacd.pid
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Resize method: none
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: No clipboard line-ending
normalization specified. Defaulting to preserving the format of all line
endings.
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: User
"@3f6a7562-58fb-46e8-a093-15a36fc8a8de" joined connection
"$85dbdc37-134b-43b6-8ea3-374137e1ed01" (1 users now present)
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Recording of session will be
saved to "/opt/guacamole/recording/screen/992382480584 -
mx-asg-db-connect-j13555-j13555-ap-southeast-3 - i-0b77f582013cbc860//202505>
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "base"
May 12 04:59:40 ac152d4d39a4432 guacd[1844190]: Loading keymap "en-us-qwerty"
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: RDP server closed/refused
connection: Disconnected.
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: User
"@3f6a7562-58fb-46e8-a093-15a36fc8a8de" disconnected (0 users remain)
May 12 04:59:49 ac152d4d39a4432 guacd[1844190]: Last user of connection
"$85dbdc37-134b-43b6-8ea3-374137e1ed01" disconnected
May 12 04:59:49 ac152d4d39a4432 guacd[1780450]: Connection
"$85dbdc37-134b-43b6-8ea3-374137e1ed01" removed.
Any suggestion is appreciated.
thanks
This e-mail and any attachments to it (the "Communication") is, unless
otherwise stated,confidential, may contain copyright material and is for the
use only of the intended recipient. If you receive the Communication in error,
please notify the sender immediately by return e-mail, delete the Communication
and the return e-mail, and do not read, copy, retransmit or otherwise deal with
it. Any views expressed in the Communication are those of the individual sender
only, unless expressly stated to be those of Australia and New Zealand Banking
Group Limited ABN 11 005 357 522, or any of its related entities including ANZ
Bank New Zealand Limited (together "ANZ"). ANZ does not accept liability in
connection with the integrity of or errors in the Communication, computer
virus, data corruption, interference or delay arising from or in respect of the
Communication.
