-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Fady,
On 2/7/17 10:53 AM, Fady Haikal wrote:
Process PID Protocol local address local port
Remote Address State Tomcat8.exe 8160 TCP
imal14-app2 4000 imal14-app1.albaraka.com.sd ESTABLISHED
Stupid question: was this working in the past, and it stopped working?
Or are you trying to get this working for the first time?
- -chris
On Tue, Feb 7, 2017 at 5:46 PM, Fady Haikal <fadyhai...@gmail.com>
wrote:
Yes there is a ESTABLISHED connection, the replication of
sessions is working fine (port 4000 is for tomcat cluster) but we
also faced this error on the log file
On Tue, Feb 7, 2017 at 5:44 PM, André Warnier (tomcat)
<a...@ice-sa.com> wrote:
On 07.02.2017 16:24, Fady Haikal wrote:
Hi, telnet IP 4000 is working fine, i installed a tool for
network monitoring at the level of IP and Port and i didnt
see any disconnection,
but did you see a *connection* ? I mean, on the pinging node,
if you use the Windows "netstat" program, for example as
netstat -aon -p TCP you should see a list of connections in the
ESTABLISHED state, of which one of the IP/ports should be your
target IP:4000 (in the "remote" column). And on the pinged
node, this port :4000 should be in the "local" column, in
LISTEN mode (and also probably one in the ESTABLISHED state, if
they agree.)
Is that the case ?
and yes i'm sure that no firewall is enabled.
I saw some strange think on the server that I have tried to
ping the multicast IP (228.0.0.4) and i get reply from
different IPs in the network, i don't know why and how i get
those IPs, after checking with the network team they told me
that those IPs are related to the SAN storage taking into
consideration that the Tomcat servers are not connected in
anyway to that SUN storage.
On Tue, Feb 7, 2017 at 4:51 PM, André Warnier (tomcat)
<a...@ice-sa.com> wrote:
Hi.
This is for the Tomcat/Tribes experts on the list.
I know nothing of Tribes, but the on-line documentation
seems to say that the communication happens over TCP and
that the protocol used is not encrypted. Fady previously
tried a standard "ping" and a "telnet" between the two
nodes, and that is the base for him mentioning that "there
is no network disconnection" between the nodes.
Nevertheless, the calling pinging node seems to say that it
times out without getting a response fom the target node.
There is evidently a contradiction there. So this could
still be some kind of network issue.
Considering that the protocol command for this "ping"
should be known by someone here, would it not be possible
to imagine a little program in some scripting language (or
even java, God forbid), which would open a TCP channel with
the target node IP/port, send such a "ping" message, wait
for a response and report the result ? That would at least
confirm/deny that the problem is with the network.
The log below does not for example say if the error happens
when opening the TCP communication channel, or after
sending the ping message on it, (Of course, testing the TCP
open could be done with "telnet IP 4000", but I don't know
if Fady tried this). Maybe tribes also already contains
some löw-level debugging options ? wireshark maybe another
option, but it has quite a learning curve. And this is on
Windows.
By the way Fady, are you sure that your "Windows Firewall
with Enhanced Security" is not just dropping TCP packets
to/from port 40xx (or from "java.exe") ? There are some
"network policies" there which can have wide-ranging
side-effects.
On 07.02.2017 14:42, Fady Haikal wrote:
Hi, issue still not fixed. Tomcat session replication is
not able to replicate the key from node to node, please
find below the error, taking into consideration that
there is no network disconnection between 2 nodes
07-Feb-2017 16:36:06.186 SEVERE [http-nio-8080-exec-8]
org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
fo
Unable to replicate backup
key:58291D242C742A8A4B1657BA42C831A4.TomcatNode2 to
backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10
,
114, 43, 102}:4000,{10, 114, 43, 102},4000, alive=68841350,
securePort=-1, UDP Port=-1, id={85 5 -62 -66 106 -12 64
12 -102 -14 -85 -87 15 9 -51 -112 }, payload={},
command={}, domain={}, ]. Reason:Operation has timed
out(3000 ms.).; Faulty members:tcp://{10, 114, 43,
102}:4000; org.apache.catalina.tribes.ChannelException:
Operation has timed out(3000 ms.).; Faulty
members:tcp://{10, 114, 43, 102}:4000; at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMe
ssage(ParallelNioSender.java:108)
at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sen
dMessage(PooledParallelSender.java:48)
at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendM
essage(ReplicationTransmitter.java:54)
at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(C
hannelCoordinator.java:82)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.MessageDispatchInte
rceptor.sendMessage(MessageDispatchInterceptor.java:81)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.
sendMessage(TcpFailureDetector.java:93)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
ava:233)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
ava:186)
at
org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
fo(LazyReplicatedMap.java:170)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
ctReplicatedMap.java:1040)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
ctReplicatedMap.java:1024)
at org.apache.catalina.session.ManagerBase.add(ManagerBase.java:647)
at
org.apache.catalina.session.StandardSession.setId(StandardSession
.java:374)
at
org.apache.catalina.ha.session.DeltaSession.setId(DeltaSession.ja
va:279)
at
org.apache.catalina.session.ManagerBase.createSession(ManagerBase
.java:708)
at org.apache.catalina.connector.Request.doGetSession(Request.java:2936)
at
org.apache.catalina.connector.Request.getSession(Request.java:226
0)
at
org.apache.catalina.connector.RequestFacade.getSession(RequestFac
ade.java:895)
at
javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServl
etRequestWrapper.java:231)
at
org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
ationHttpRequest.java:568)
at
org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
ationHttpRequest.java:513)
at
org.apache.jasper.runtime.PageContextImpl.initialize(PageContextI
mpl.java:137)
at
org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(J
spFactoryImpl.java:109)
at
org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactor
yImpl.java:60)
at
org.apache.jsp.WEB_002dINF.jsp._401_jsp._jspService(_401_jsp.java
:100)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWra
pper.java:438)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.ja
va:396)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:291)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicat
ionFilterChain.java:206)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(Application
Dispatcher.java:719)
at
org.apache.catalina.core.ApplicationDispatcher.processRequest(App
licationDispatcher.java:467)
at
org.apache.catalina.core.ApplicationDispatcher.doForward(Applicat
ionDispatcher.java:390)
at
org.apache.catalina.core.ApplicationDispatcher.forward(Applicatio
nDispatcher.java:317)
at
org.apache.catalina.core.StandardHostValve.custom(StandardHostVal
ve.java:445)
at
org.apache.catalina.core.StandardHostValve.status(StandardHostVal
ve.java:304)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostVal
ve.java:181)
at
org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRout
eBinderValve.java:194)
at
org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationVal
ve.java:318)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportVal
ve.java:79)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(Stuck
ThreadDetectionValve.java:208)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abstract
AccessLogValve.java:616)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngin
eValve.java:88)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter
.java:521)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(Abstract
Http11Processor.java:1096)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.proc
ess(AbstractProtocol.java:674)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioE
ndpoint.java:1500)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEnd
point.java:1456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source) at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Ta
skThread.java:61)
at java.lang.Thread.run(Unknown Source)
On Tue, Feb 7, 2017 at 1:02 PM, Kreuser, Peter
<pkreu...@airplus.com> wrote:
Fady,
Sorry for top posting.
If I remember correctly, the Cluster Element goes into
the Container and not the Host. Plus I see in our
(working) case, a DeltaManager and a
JvmRouteSessionIDBinderListener
<Manager
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/> ...
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinde
rListener"/>
Besides this, only ports, limits and values are different.
You may want to filter out the replication for static
resources as gifs jpg or css. <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;
" />
Best regards
Peter
below is the server.xml configuration, as mentioened
earlier the issue is related to the cluster
configuration, and as per my research i can see that
some users are facing the same issue but i didnt
found the solution of it
<?xml version='1.0' encoding='utf-8'?> <!-- Licensed
to the Apache Software Foundation (ASF) under one or
more contributor license agreements. See the NOTICE
file distributed with this work for additional
information regarding copyright ownership. The ASF
licenses this file to You under the Apache License,
Version 2.0 (the "License"); you may not use this
file except in compliance with the License. You may
obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in
writing, software distributed under the License is
distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language
governing permissions and limitations under the
License. --> <!-- Note: A "Server" is not itself a
"Container", so you may not define subcomponents such
as "Valves" at this level. Documentation at
/docs/config/server.html --> <Server port="8005"
shutdown="SHUTDOWN"> <Listener
className="org.apache.catalina.startup.VersionLoggerListener"
/> <!-- Security listener. Documentation at
/docs/config/listeners.html <Listener
className="org.apache.catalina.security.SecurityListener"
/>
--> <!--APR library loader. Documentation at
/docs/apr.html --> <Listener
className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!-- Prevent memory leaks due to use of particular
java/javax APIs--> <Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionList
ener"
/>
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleL
istener"
/>
<Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionLi
stener"
/>
<!-- Global JNDI resources Documentation at
/docs/jndi-resources-howto.html -->
<GlobalNamingResources> <!-- Editable user database
that can also be used by UserDatabaseRealm to
authenticate users --> <Resource name="UserDatabase"
auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and
saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more
"Connectors" that share a single "Container" Note: A
"Service" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html -->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can
define one or more named thread pools--> <!--
<Executor name="tomcatThreadPool"
namePrefix="catalina-exec-" maxThreads="150"
minSpareThreads="4"/> -->
<!-- A "Connector" represents an endpoint by which
requests are received and responses are returned.
Documentation at : Java HTTP Connector:
/docs/config/http.html (blocking & non-blocking) Java
AJP Connector: /docs/config/ajp.html APR (HTTP/AJP)
Connector: /docs/apr.html Define a non-SSL/TLS
HTTP/1.1 Connector on port 8080 --> <Connector
port="8080" protocol="HTTP/1.1"
connectionTimeout="60000" maxThreads="500"
minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" redirectPort="8443" /> <!-- A
"Connector" using the shared thread pool--> <!--
<Connector executor="tomcatThreadPool" port="8080"
protocol="HTTP/1.1" connectionTimeout="60000"
redirectPort="8443" /> --> <!-- Define a SSL/TLS
HTTP/1.1 Connector on port 8443 This connector uses
the NIO implementation that requires the JSSE style
configuration. When using the APR/native
implementation, the OpenSSL style configuration is
required as described in the APR/native documentation
--> <!-- <Connector port="8443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https"
secure="true" clientAuth="false" sslProtocol="TLS"
/> -->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3"
redirectPort="8443" />
<!-- An Engine represents the entry point (within
Catalina) that processes every request. The Engine
implementation for Tomcat stand alone analyzes the
HTTP headers included with the request, and passes
them on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support
load-balancing via AJP ie :
--> <Engine name="Catalina" defaultHost="localhost"
jvmRoute="TomcatNode1"> <!-- Use the LockOutRealm to
prevent attempts to guess user passwords via a
brute-force attack --> <Realm
className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in
the global JNDI resources under the key
"UserDatabase". Any edits that are performed against
this UserDatabase are immediately available for use
by the Realm. --> <Realm
className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
startStopThreads="0"> <!--For clustering, please take
a look at documentation at: /docs/cluster-howto.html
(simple how to) /docs/config/cluster.html (reference
documentation) --> <!-- <Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
- -->
<Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="4">
<Channel
className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564" frequency="500" dropTime="9000"/>
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver
"
address="auto"
port="4000" autoBind="100" selectorTimeout="5000"
maxThreads="6"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTran
smitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParal
lelSender"/>
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFai
lureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.Messag
eDispatch15Interceptor"/>
</Channel>
<Valve
className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/
<Deployer
className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="D:/imaljava/TomcatNode1/tmp/war-temp/"
deployDir="D:/imaljava/TomcatNode1/tmp/war-deploy/"
watchDir="D:/imaljava/TomcatNode1/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListene
r"/>
</Cluster>
<!-- SingleSignOn valve, share authentication between
web applications Documentation at:
/docs/config/valve.html --> <!-- <Valve
className="org.apache.catalina.authenticator.SingleSignOn"
/> -->
<!-- Access log processes all example. Documentation
at: /docs/config/valve.html Note: The pattern used is
equivalent to using pattern="common" --> <Valve
className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" /> <Valve
className="org.apache.catalina.valves.StuckThreadDetectionValve
"
threshold="900" />
</Host> </Engine> </Service> </Server>
On Mon, Feb 6, 2017 at 6:51 PM, André Warnier
(tomcat) <a...@ice-sa.com> wrote:
On 06.02.2017 17:45, Fady Haikal wrote:
Hi, What is the host OS ? Windows Server 2012
What is the Tomcat version ? Apache
Tomcat/8.0.30
Is this problem new ? was this working before ?
how long ? Since cluster implementation
I still don't know tribes, but then my non-educated
guess at this point would be that there is
something wrong in your configuration. Can you
copy/paste it here ? (remove sensible things like
passwords, public IP addresses etc..)(but not to
the point of making it uncheckable).
Then maybe some tribes-specialist can take over ?
Is there actually something listening on that
address/port ? Tomcat cluster
the Port 4000 is listening and there is no
disconnection between 2 nodes ping and telnet are
OK
On Mon, Feb 6, 2017 at 6:42 PM, André Warnier
(tomcat) <a...@ice-sa.com> wrote:
On 06.02.2017 17:24, Fady Haikal wrote:
Plz can i get some help here? This issue is
still occurring and it's filling the log file
in the Production server
Regards, Fady
Hi. If you want quick answers, you should
provide more information. What is the host OS
? What is the Tomcat version ? Is this problem
new ? was this working before ? how long ?
I do not know tribes at all, but according to
the logfile below, it seems that something is
trying to "ping" the address 10.114.43.103,
port 4000, and never getting a response (or at
least not within 3000ms). Is there actually
something listening on that address/port ? The
"netstat" command (available both on Linux and
Windows) can tell you. If there is something
listening there, can it respond to whatever is
ping-ing it ? (routing, firewall, ..)
On Mon, Feb 6, 2017 at 8:52 AM, Fady Haikal
<fadyhai...@gmail.com> wrote:
Guys, we are facing the below errors in
Tomcat cluster, please advise
06-Feb-2017 01:14:20.718 SEVERE
[GroupChannel-Heartbeat-1]
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
rtbeat
Unable to send AbstractReplicatedMap.ping message
org.apache.catalina.tribes.ChannelException:
Operation has timed out(3000 ms.).; Faulty
members:tcp://{10, 114, 43, 103}:4000; at
org.apache.catalina.tribes.transport.nio.ParallelNioSender
.sendMessage(ParallelNioSender.java:108)
at
org.apache.catalina.tribes.transport.nio.PooledParallelSen
der.sendMessage(PooledParallelSender.java:48)
at
org.apache.catalina.tribes.transport.ReplicationTransmitte
r.sendMessage(ReplicationTransmitter.java:54)
at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMe
ssage(ChannelCoordinator.java:82)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.MessageDispa
tchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.interceptors.TcpFailureDe
tector.sendMessage(TcpFailureDetector.java:93)
at
org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
annel.java:233)
at
org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
annel.java:186)
at
org.apache.catalina.tribes.group.RpcChannel.send(RpcChanne
l.java:99)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.pin
g(AbstractReplicatedMap.java:267)
at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
rtbeat(AbstractReplicatedMap.java:885)
at
org.apache.catalina.tribes.group.GroupChannel.heartbeat(Gr
oupChannel.java:161)
at
org.apache.catalina.tribes.group.GroupChannel$HeartbeatThr
ead.run(GroupChannel.java:697)
06-Feb-2017 01:20:51.437 SEVERE [NioReceiver]
org.apache.catalina.tribes.transport.nio.NioReceiver.liste
n
Unable
to process request in NioReceiver
java.io.IOException: A non-blocking socket
operation could not be completed
immediately at
sun.nio.ch.SocketDispatcher.close0(Native
Method) at
sun.nio.ch.SocketDispatcher.close(Unknown
Source) at
sun.nio.ch.SocketChannelImpl.kill(Unknown
Source) at
sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown
Source) at
sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown
Source) at
sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown
Source) at
sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown
Source) at
sun.nio.ch.SelectorImpl.select(Unknown
Source) at
org.apache.catalina.tribes.transport.nio.NioReceiver.liste
n(NioReceiver.java:272)
at
org.apache.catalina.tribes.transport.nio.NioReceiver.run(N
ioReceiver.java:425)
at java.lang.Thread.run(Unknown Source)
Regards, Fady
-----------------------------------------------------------
- ----------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
------------------------------------------------------------
- ---------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
-------------------------------------------------------------
- --------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
--------------------------------------------------------------
- -------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
---------------------------------------------------------------
- ------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
-----------------------------------------------------------------
- ----
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
------------------------------------------------------------------
- ---
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
-------------------------------------------------------------------
- --
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:
users-h...@tomcat.apache.org
--------------------------------------------------------------------
- -
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJYmfN4AAoJEBzwKT+lPKRYZuIQAI2NXtOMNcNpd88y2rXRZFCK
p3YN8OBo07UMkT+N6OgmkCrWBGRqIk9SZSUugnWP7N1n5nOFpaio/I2n/2dGOXDB
oYfPA3KVkWaMQenOAujjyodi7Q7ABUurJ07V+D0zfBtPpqE7up9/edl94pq8znuR
KGp1eEKeMlMiZAS9vm5/0ifhhMIDGfE/Y3xjqf+Kb5gHXlA5BrOrlo7+UX26uuqr
qIIaobTCKYaQvYE5HP+9FyOsXCd1oicKz9JIPAoc8A7Yj//n+5Ogtoczyu+mSzxL
X7/JSRdywbyfFo+1bOaecxmx9Bh9UgyziEh2AuXXMGwf17a58ciN8VCqtJcJC3HO
rffZtc9sqm/Tyt5tvo5PgPbpTrvsjeZXMrxUx5QuOjWlkoVOPptACB4LATVmXaoM
lHGw0OBCQgNViI5LXMs59QSdj9/J7tzN+0AakFmO0GM2fHjiFzgHTLGa5QA9yO7b
wIDpCQj3cDFig+4z95ZFHcGTmcUd9u5a6dO/4BPzFvsR/r6kfmH7qV1mwVu1TkEH
4B6EO5AwCwZJNM2asxFktswNoI4Wl2pP7ZjdxW8tIXchJco0iHn/VhnO1kPeNI8x
ws6KXUpZqPp4ylLWYKTLHg9fMc9Ffn6u6jR09o9amEIWnSlinA4R+n9VztWiBR45
h9q0x3f+aKuQp2IE0UBE
=WoKC
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
