On 07.02.2017 16:24, Fady Haikal wrote:
Hi,
telnet IP 4000 is working fine, i installed a tool for network
monitoring at the level of IP and Port and i didnt see any
disconnection,
but did you see a *connection* ?
I mean, on the pinging node, if you use the Windows "netstat" program, for
example as
netstat -aon -p TCP
you should see a list of connections in the ESTABLISHED state, of which one of the
IP/ports should be your target IP:4000 (in the "remote" column).
And on the pinged node, this port :4000 should be in the "local" column, in
LISTEN mode
(and also probably one in the ESTABLISHED state, if they agree.)
Is that the case ?
and yes i'm sure that no firewall is enabled.
I saw some strange think on the server that I have tried to ping the
multicast IP (228.0.0.4) and i get reply from different IPs in the
network, i don't know why and how i get those IPs, after checking with
the network team they told me that those IPs are related to the SAN
storage taking into consideration that the Tomcat servers are not
connected in anyway to that SUN storage.
On Tue, Feb 7, 2017 at 4:51 PM, André Warnier (tomcat) <a...@ice-sa.com> wrote:
Hi.
This is for the Tomcat/Tribes experts on the list.
I know nothing of Tribes, but the on-line documentation seems to say that
the communication happens over TCP and that the protocol used is not
encrypted.
Fady previously tried a standard "ping" and a "telnet" between the two
nodes, and that is the base for him mentioning that "there is no network
disconnection" between the nodes.
Nevertheless, the calling pinging node seems to say that it times out
without getting a response fom the target node. There is evidently a
contradiction there.
So this could still be some kind of network issue.
Considering that the protocol command for this "ping" should be known by
someone here, would it not be possible to imagine a little program in some
scripting language (or even java, God forbid), which would open a TCP
channel with the target node IP/port, send such a "ping" message, wait for a
response and report the result ?
That would at least confirm/deny that the problem is with the network.
The log below does not for example say if the error happens when opening the
TCP communication channel, or after sending the ping message on it,
(Of course, testing the TCP open could be done with "telnet IP 4000", but I
don't know if Fady tried this).
Maybe tribes also already contains some löw-level debugging options ?
wireshark maybe another option, but it has quite a learning curve.
And this is on Windows.
By the way Fady, are you sure that your "Windows Firewall with Enhanced
Security" is not just dropping TCP packets to/from port 40xx (or from
"java.exe") ? There are some "network policies" there which can have
wide-ranging side-effects.
On 07.02.2017 14:42, Fady Haikal wrote:
Hi, issue still not fixed. Tomcat session replication is not able to
replicate the key from node to node, please find below the error,
taking into consideration that there is no network disconnection
between 2 nodes
07-Feb-2017 16:36:06.186 SEVERE [http-nio-8080-exec-8]
org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryInfo
Unable to replicate backup
key:58291D242C742A8A4B1657BA42C831A4.TomcatNode2 to
backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10,
114, 43, 102}:4000,{10, 114, 43, 102},4000, alive=68841350,
securePort=-1, UDP Port=-1, id={85 5 -62 -66 106 -12 64 12 -102 -14
-85 -87 15 9 -51 -112 }, payload={}, command={}, domain={}, ].
Reason:Operation has timed out(3000 ms.).; Faulty members:tcp://{10,
114, 43, 102}:4000;
org.apache.catalina.tribes.ChannelException: Operation has timed
out(3000 ms.).; Faulty members:tcp://{10, 114, 43, 102}:4000;
at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:108)
at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:48)
at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:54)
at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:82)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:93)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:233)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:186)
at
org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryInfo(LazyReplicatedMap.java:170)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(AbstractReplicatedMap.java:1040)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(AbstractReplicatedMap.java:1024)
at org.apache.catalina.session.ManagerBase.add(ManagerBase.java:647)
at
org.apache.catalina.session.StandardSession.setId(StandardSession.java:374)
at
org.apache.catalina.ha.session.DeltaSession.setId(DeltaSession.java:279)
at
org.apache.catalina.session.ManagerBase.createSession(ManagerBase.java:708)
at org.apache.catalina.connector.Request.doGetSession(Request.java:2936)
at org.apache.catalina.connector.Request.getSession(Request.java:2260)
at
org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:895)
at
javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
at
org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:568)
at
org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:513)
at
org.apache.jasper.runtime.PageContextImpl.initialize(PageContextImpl.java:137)
at
org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(JspFactoryImpl.java:109)
at
org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactoryImpl.java:60)
at org.apache.jsp.WEB_002dINF.jsp._401_jsp._jspService(_401_jsp.java:100)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:438)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:396)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:719)
at
org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:467)
at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:390)
at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:317)
at
org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:445)
at
org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:304)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:181)
at
org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:194)
at
org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:318)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:208)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)
On Tue, Feb 7, 2017 at 1:02 PM, Kreuser, Peter <pkreu...@airplus.com>
wrote:
Fady,
Sorry for top posting.
If I remember correctly, the Cluster Element goes into the Container and
not the Host.
Plus I see in our (working) case, a DeltaManager and a
JvmRouteSessionIDBinderListener
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
...
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
Besides this, only ports, limits and values are different.
You may want to filter out the replication for static resources as gifs
jpg or css.
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" />
Best regards
Peter
below is the server.xml configuration, as mentioened earlier the issue
is related to the cluster configuration, and as per my research i can
see that some users are facing the same issue but i didnt found the
solution of it
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version
2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<Listener
className="org.apache.catalina.startup.VersionLoggerListener" />
<!-- Security listener. Documentation at /docs/config/listeners.html
<Listener className="org.apache.catalina.security.SecurityListener"
/>
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
/>
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"
/>
<Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener"
/>
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that
share
a single "Container" Note: A "Service" is not itself a
"Container",
so you may not define subcomponents such as "Valves" at this
level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one
or more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are
received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking &
non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="60000" maxThreads="500"
minSpareThreads="25" maxSpareThreads="75" enableLookups="false"
disableUploadTimeout="true" acceptCount="100" redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="60000"
redirectPort="8443" />
-->
<!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
This connector uses the NIO implementation that requires the
JSSE
style configuration. When using the APR/native implementation,
the
OpenSSL style configuration is required as described in the
APR/native
documentation -->
<!--
<Connector port="8443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https"
secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that
processes
every request. The Engine implementation for Tomcat stand
alone
analyzes the HTTP headers included with the request, and
passes them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
-->
<Engine name="Catalina" defaultHost="localhost"
jvmRoute="TomcatNode1">
<!-- Use the LockOutRealm to prevent attempts to guess user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in the global
JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are
immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true" startStopThreads="0">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<!--
<Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="4">
<Channel
className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="9000"/>
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve
className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer
className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="D:/imaljava/TomcatNode1/tmp/war-temp/"
deployDir="D:/imaljava/TomcatNode1/tmp/war-deploy/"
watchDir="D:/imaljava/TomcatNode1/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
<!-- SingleSignOn valve, share authentication between web
applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve
className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using
pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Valve className="org.apache.catalina.valves.StuckThreadDetectionValve"
threshold="900" />
</Host>
</Engine>
</Service>
</Server>
On Mon, Feb 6, 2017 at 6:51 PM, André Warnier (tomcat) <a...@ice-sa.com>
wrote:
On 06.02.2017 17:45, Fady Haikal wrote:
Hi,
What is the host OS ? Windows Server 2012
What is the Tomcat version ? Apache Tomcat/8.0.30
Is this problem new ? was this working before ? how long ? Since
cluster implementation
I still don't know tribes, but then my non-educated guess at this point
would be that there is something wrong in your configuration.
Can you copy/paste it here ? (remove sensible things like passwords,
public
IP addresses etc..)(but not to the point of making it uncheckable).
Then maybe some tribes-specialist can take over ?
Is there actually something listening on that address/port ? Tomcat
cluster
the Port 4000 is listening and there is no disconnection between 2
nodes ping and telnet are OK
On Mon, Feb 6, 2017 at 6:42 PM, André Warnier (tomcat) <a...@ice-sa.com>
wrote:
On 06.02.2017 17:24, Fady Haikal wrote:
Plz can i get some help here?
This issue is still occurring and it's filling the log file in the
Production server
Regards,
Fady
Hi.
If you want quick answers, you should provide more information.
What is the host OS ?
What is the Tomcat version ?
Is this problem new ? was this working before ? how long ?
I do not know tribes at all, but according to the logfile below, it
seems
that something is trying to "ping" the address 10.114.43.103, port
4000,
and
never getting a response (or at least not within 3000ms).
Is there actually something listening on that address/port ?
The "netstat" command (available both on Linux and Windows) can tell
you.
If there is something listening there, can it respond to whatever is
ping-ing it ?
(routing, firewall, ..)
On Mon, Feb 6, 2017 at 8:52 AM, Fady Haikal <fadyhai...@gmail.com>
wrote:
Guys, we are facing the below errors in Tomcat cluster, please
advise
06-Feb-2017 01:14:20.718 SEVERE [GroupChannel-Heartbeat-1]
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.heartbeat
Unable to send AbstractReplicatedMap.ping message
org.apache.catalina.tribes.ChannelException: Operation has
timed
out(3000 ms.).; Faulty members:tcp://{10, 114, 43, 103}:4000;
at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:108)
at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:48)
at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:54)
at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:82)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:93)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:233)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:186)
at
org.apache.catalina.tribes.group.RpcChannel.send(RpcChannel.java:99)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.ping(AbstractReplicatedMap.java:267)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.heartbeat(AbstractReplicatedMap.java:885)
at
org.apache.catalina.tribes.group.GroupChannel.heartbeat(GroupChannel.java:161)
at
org.apache.catalina.tribes.group.GroupChannel$HeartbeatThread.run(GroupChannel.java:697)
06-Feb-2017 01:20:51.437 SEVERE [NioReceiver]
org.apache.catalina.tribes.transport.nio.NioReceiver.listen Unable
to
process request in NioReceiver
java.io.IOException: A non-blocking socket operation could not
be
completed immediately
at sun.nio.ch.SocketDispatcher.close0(Native Method)
at sun.nio.ch.SocketDispatcher.close(Unknown Source)
at sun.nio.ch.SocketChannelImpl.kill(Unknown Source)
at sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown Source)
at sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown Source)
at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
at sun.nio.ch.SelectorImpl.select(Unknown Source)
at
org.apache.catalina.tribes.transport.nio.NioReceiver.listen(NioReceiver.java:272)
at
org.apache.catalina.tribes.transport.nio.NioReceiver.run(NioReceiver.java:425)
at java.lang.Thread.run(Unknown Source)
Regards,
Fady
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
