-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Fady,
On 2/7/17 10:53 AM, Fady Haikal wrote:
> Process PID Protocol local address local port
> Remote Address State Tomcat8.exe 8160 TCP
> imal14-app2 4000 imal14-app1.albaraka.com.sd ESTABLISHED
Stupid question: was this working in the past, and it stopped working?
Or are you trying to get this working for the first time?
- -chris
> On Tue, Feb 7, 2017 at 5:46 PM, Fady Haikal <fadyhai...@gmail.com>
> wrote:
>> Yes there is a ESTABLISHED connection, the replication of
>> sessions is working fine (port 4000 is for tomcat cluster) but we
>> also faced this error on the log file
>>
>> On Tue, Feb 7, 2017 at 5:44 PM, André Warnier (tomcat)
>> <a...@ice-sa.com> wrote:
>>> On 07.02.2017 16:24, Fady Haikal wrote:
>>>>
>>>> Hi, telnet IP 4000 is working fine, i installed a tool for
>>>> network monitoring at the level of IP and Port and i didnt
>>>> see any disconnection,
>>>
>>>
>>> but did you see a *connection* ? I mean, on the pinging node,
>>> if you use the Windows "netstat" program, for example as
>>> netstat -aon -p TCP you should see a list of connections in the
>>> ESTABLISHED state, of which one of the IP/ports should be your
>>> target IP:4000 (in the "remote" column). And on the pinged
>>> node, this port :4000 should be in the "local" column, in
>>> LISTEN mode (and also probably one in the ESTABLISHED state, if
>>> they agree.)
>>>
>>> Is that the case ?
>>>
>>>
>>>
>>> and yes i'm sure that no firewall is enabled.
>>>>
>>>>
>>>> I saw some strange think on the server that I have tried to
>>>> ping the multicast IP (228.0.0.4) and i get reply from
>>>> different IPs in the network, i don't know why and how i get
>>>> those IPs, after checking with the network team they told me
>>>> that those IPs are related to the SAN storage taking into
>>>> consideration that the Tomcat servers are not connected in
>>>> anyway to that SUN storage.
>>>>
>>>>
>>>> On Tue, Feb 7, 2017 at 4:51 PM, André Warnier (tomcat)
>>>> <a...@ice-sa.com> wrote:
>>>>>
>>>>> Hi.
>>>>>
>>>>> This is for the Tomcat/Tribes experts on the list.
>>>>>
>>>>> I know nothing of Tribes, but the on-line documentation
>>>>> seems to say that the communication happens over TCP and
>>>>> that the protocol used is not encrypted. Fady previously
>>>>> tried a standard "ping" and a "telnet" between the two
>>>>> nodes, and that is the base for him mentioning that "there
>>>>> is no network disconnection" between the nodes.
>>>>> Nevertheless, the calling pinging node seems to say that it
>>>>> times out without getting a response fom the target node.
>>>>> There is evidently a contradiction there. So this could
>>>>> still be some kind of network issue.
>>>>>
>>>>> Considering that the protocol command for this "ping"
>>>>> should be known by someone here, would it not be possible
>>>>> to imagine a little program in some scripting language (or
>>>>> even java, God forbid), which would open a TCP channel with
>>>>> the target node IP/port, send such a "ping" message, wait
>>>>> for a response and report the result ? That would at least
>>>>> confirm/deny that the problem is with the network.
>>>>>
>>>>> The log below does not for example say if the error happens
>>>>> when opening the TCP communication channel, or after
>>>>> sending the ping message on it, (Of course, testing the TCP
>>>>> open could be done with "telnet IP 4000", but I don't know
>>>>> if Fady tried this). Maybe tribes also already contains
>>>>> some löw-level debugging options ? wireshark maybe another
>>>>> option, but it has quite a learning curve. And this is on
>>>>> Windows.
>>>>>
>>>>> By the way Fady, are you sure that your "Windows Firewall
>>>>> with Enhanced Security" is not just dropping TCP packets
>>>>> to/from port 40xx (or from "java.exe") ? There are some
>>>>> "network policies" there which can have wide-ranging
>>>>> side-effects.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 07.02.2017 14:42, Fady Haikal wrote:
>>>>>>
>>>>>>
>>>>>> Hi, issue still not fixed. Tomcat session replication is
>>>>>> not able to replicate the key from node to node, please
>>>>>> find below the error, taking into consideration that
>>>>>> there is no network disconnection between 2 nodes
>>>>>>
>>>>>>
>>>>>> 07-Feb-2017 16:36:06.186 SEVERE [http-nio-8080-exec-8]
>>>>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
fo
>>>>>>
>>>>>>
Unable to replicate backup
>>>>>> key:58291D242C742A8A4B1657BA42C831A4.TomcatNode2 to
>>>>>> backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10
,
>>>>>>
>>>>>>
114, 43, 102}:4000,{10, 114, 43, 102},4000, alive=68841350,
>>>>>> securePort=-1, UDP Port=-1, id={85 5 -62 -66 106 -12 64
>>>>>> 12 -102 -14 -85 -87 15 9 -51 -112 }, payload={},
>>>>>> command={}, domain={}, ]. Reason:Operation has timed
>>>>>> out(3000 ms.).; Faulty members:tcp://{10, 114, 43,
>>>>>> 102}:4000; org.apache.catalina.tribes.ChannelException:
>>>>>> Operation has timed out(3000 ms.).; Faulty
>>>>>> members:tcp://{10, 114, 43, 102}:4000; at
>>>>>>
>>>>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMe
ssage(ParallelNioSender.java:108)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sen
dMessage(PooledParallelSender.java:48)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendM
essage(ReplicationTransmitter.java:54)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(C
hannelCoordinator.java:82)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.interceptors.MessageDispatchInte
rceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.
sendMessage(TcpFailureDetector.java:93)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
ge(ChannelInterceptorBase.java:76)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
ava:233)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
ava:186)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
fo(LazyReplicatedMap.java:170)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
ctReplicatedMap.java:1040)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
ctReplicatedMap.java:1024)
>>>>>>
>>>>>>
at org.apache.catalina.session.ManagerBase.add(ManagerBase.java:647)
>>>>>> at
>>>>>>
>>>>>> org.apache.catalina.session.StandardSession.setId(StandardSession
.java:374)
>>>>>>
>>>>>>
at
>>>>>> org.apache.catalina.ha.session.DeltaSession.setId(DeltaSession.ja
va:279)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.session.ManagerBase.createSession(ManagerBase
.java:708)
>>>>>>
>>>>>>
at org.apache.catalina.connector.Request.doGetSession(Request.java:2936)
>>>>>> at
>>>>>> org.apache.catalina.connector.Request.getSession(Request.java:226
0)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.connector.RequestFacade.getSession(RequestFac
ade.java:895)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServl
etRequestWrapper.java:231)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
ationHttpRequest.java:568)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
ationHttpRequest.java:513)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.jasper.runtime.PageContextImpl.initialize(PageContextI
mpl.java:137)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(J
spFactoryImpl.java:109)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactor
yImpl.java:60)
>>>>>>
>>>>>>
at
>>>>>> org.apache.jsp.WEB_002dINF.jsp._401_jsp._jspService(_401_jsp.java
:100)
>>>>>>
>>>>>>
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>>>>> at
>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWra
pper.java:438)
>>>>>>
>>>>>>
at
>>>>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.ja
va:396)
>>>>>>
>>>>>>
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
>>>>>> at
>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:291)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicat
ionFilterChain.java:206)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationDispatcher.invoke(Application
Dispatcher.java:719)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationDispatcher.processRequest(App
licationDispatcher.java:467)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationDispatcher.doForward(Applicat
ionDispatcher.java:390)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.ApplicationDispatcher.forward(Applicatio
nDispatcher.java:317)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.StandardHostValve.custom(StandardHostVal
ve.java:445)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.StandardHostValve.status(StandardHostVal
ve.java:304)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostVal
ve.java:181)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRout
eBinderValve.java:194)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationVal
ve.java:318)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportVal
ve.java:79)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.valves.StuckThreadDetectionValve.invoke(Stuck
ThreadDetectionValve.java:208)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abstract
AccessLogValve.java:616)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngin
eValve.java:88)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter
.java:521)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(Abstract
Http11Processor.java:1096)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.proc
ess(AbstractProtocol.java:674)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioE
ndpoint.java:1500)
>>>>>>
>>>>>>
at
>>>>>>
>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEnd
point.java:1456)
>>>>>>
>>>>>>
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>>>>>> Source) at
>>>>>>
>>>>>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Ta
skThread.java:61)
>>>>>>
>>>>>>
at java.lang.Thread.run(Unknown Source)
>>>>>>
>>>>>> On Tue, Feb 7, 2017 at 1:02 PM, Kreuser, Peter
>>>>>> <pkreu...@airplus.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Fady,
>>>>>>>
>>>>>>> Sorry for top posting.
>>>>>>>
>>>>>>> If I remember correctly, the Cluster Element goes into
>>>>>>> the Container and not the Host. Plus I see in our
>>>>>>> (working) case, a DeltaManager and a
>>>>>>> JvmRouteSessionIDBinderListener
>>>>>>>
>>>>>>> <Manager
>>>>>>> className="org.apache.catalina.ha.session.DeltaManager"
>>>>>>>
>>>>>>>
expireSessionsOnShutdown="false"
>>>>>>> notifyListenersOnReplication="true"/> ...
>>>>>>> <ClusterListener
>>>>>>>
>>>>>>> className="org.apache.catalina.ha.session.JvmRouteSessionIDBinde
rListener"/>
>>>>>>>
>>>>>>>
>>>>>>>
Besides this, only ports, limits and values are different.
>>>>>>>
>>>>>>> You may want to filter out the replication for static
>>>>>>> resources as gifs jpg or css. <Valve
>>>>>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>>>>>
>>>>>>>
>>>>>>>
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;
" />
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>> Peter
>>>>>>>
>>>>>>>> below is the server.xml configuration, as mentioened
>>>>>>>> earlier the issue is related to the cluster
>>>>>>>> configuration, and as per my research i can see that
>>>>>>>> some users are facing the same issue but i didnt
>>>>>>>> found the solution of it
>>>>>>>>
>>>>>>>>
>>>>>>>> <?xml version='1.0' encoding='utf-8'?> <!-- Licensed
>>>>>>>> to the Apache Software Foundation (ASF) under one or
>>>>>>>> more contributor license agreements. See the NOTICE
>>>>>>>> file distributed with this work for additional
>>>>>>>> information regarding copyright ownership. The ASF
>>>>>>>> licenses this file to You under the Apache License,
>>>>>>>> Version 2.0 (the "License"); you may not use this
>>>>>>>> file except in compliance with the License. You may
>>>>>>>> obtain a copy of the License at
>>>>>>>>
>>>>>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>
>>>>>>>> Unless required by applicable law or agreed to in
>>>>>>>> writing, software distributed under the License is
>>>>>>>> distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
>>>>>>>> OR CONDITIONS OF ANY KIND, either express or
>>>>>>>> implied. See the License for the specific language
>>>>>>>> governing permissions and limitations under the
>>>>>>>> License. --> <!-- Note: A "Server" is not itself a
>>>>>>>> "Container", so you may not define subcomponents such
>>>>>>>> as "Valves" at this level. Documentation at
>>>>>>>> /docs/config/server.html --> <Server port="8005"
>>>>>>>> shutdown="SHUTDOWN"> <Listener
>>>>>>>> className="org.apache.catalina.startup.VersionLoggerListener"
>>>>>>>> /> <!-- Security listener. Documentation at
>>>>>>>> /docs/config/listeners.html <Listener
>>>>>>>> className="org.apache.catalina.security.SecurityListener"
>>>>>>>>
>>>>>>>>
/>
>>>>>>>> --> <!--APR library loader. Documentation at
>>>>>>>> /docs/apr.html --> <Listener
>>>>>>>> className="org.apache.catalina.core.AprLifecycleListener"
>>>>>>>>
>>>>>>>>
SSLEngine="on" />
>>>>>>>> <!-- Prevent memory leaks due to use of particular
>>>>>>>> java/javax APIs--> <Listener
>>>>>>>> className="org.apache.catalina.core.JreMemoryLeakPreventionList
ener"
>>>>>>>>
>>>>>>>>
/>
>>>>>>>> <Listener
>>>>>>>>
>>>>>>>> className="org.apache.catalina.mbeans.GlobalResourcesLifecycleL
istener"
>>>>>>>>
>>>>>>>>
/>
>>>>>>>> <Listener
>>>>>>>> className="org.apache.catalina.core.ThreadLocalLeakPreventionLi
stener"
>>>>>>>>
>>>>>>>>
/>
>>>>>>>>
>>>>>>>> <!-- Global JNDI resources Documentation at
>>>>>>>> /docs/jndi-resources-howto.html -->
>>>>>>>> <GlobalNamingResources> <!-- Editable user database
>>>>>>>> that can also be used by UserDatabaseRealm to
>>>>>>>> authenticate users --> <Resource name="UserDatabase"
>>>>>>>> auth="Container"
>>>>>>>> type="org.apache.catalina.UserDatabase"
>>>>>>>> description="User database that can be updated and
>>>>>>>> saved"
>>>>>>>>
>>>>>>>> factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
>>>>>>>>
>>>>>>>>
pathname="conf/tomcat-users.xml" />
>>>>>>>> </GlobalNamingResources>
>>>>>>>>
>>>>>>>> <!-- A "Service" is a collection of one or more
>>>>>>>> "Connectors" that share a single "Container" Note: A
>>>>>>>> "Service" is not itself a "Container", so you may not
>>>>>>>> define subcomponents such as "Valves" at this level.
>>>>>>>> Documentation at /docs/config/service.html -->
>>>>>>>> <Service name="Catalina">
>>>>>>>>
>>>>>>>> <!--The connectors can use a shared executor, you can
>>>>>>>> define one or more named thread pools--> <!--
>>>>>>>> <Executor name="tomcatThreadPool"
>>>>>>>> namePrefix="catalina-exec-" maxThreads="150"
>>>>>>>> minSpareThreads="4"/> -->
>>>>>>>>
>>>>>>>>
>>>>>>>> <!-- A "Connector" represents an endpoint by which
>>>>>>>> requests are received and responses are returned.
>>>>>>>> Documentation at : Java HTTP Connector:
>>>>>>>> /docs/config/http.html (blocking & non-blocking) Java
>>>>>>>> AJP Connector: /docs/config/ajp.html APR (HTTP/AJP)
>>>>>>>> Connector: /docs/apr.html Define a non-SSL/TLS
>>>>>>>> HTTP/1.1 Connector on port 8080 --> <Connector
>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>> connectionTimeout="60000" maxThreads="500"
>>>>>>>> minSpareThreads="25" maxSpareThreads="75"
>>>>>>>> enableLookups="false" disableUploadTimeout="true"
>>>>>>>> acceptCount="100" redirectPort="8443" /> <!-- A
>>>>>>>> "Connector" using the shared thread pool--> <!--
>>>>>>>> <Connector executor="tomcatThreadPool" port="8080"
>>>>>>>> protocol="HTTP/1.1" connectionTimeout="60000"
>>>>>>>> redirectPort="8443" /> --> <!-- Define a SSL/TLS
>>>>>>>> HTTP/1.1 Connector on port 8443 This connector uses
>>>>>>>> the NIO implementation that requires the JSSE style
>>>>>>>> configuration. When using the APR/native
>>>>>>>> implementation, the OpenSSL style configuration is
>>>>>>>> required as described in the APR/native documentation
>>>>>>>> --> <!-- <Connector port="8443"
>>>>>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>>>>>>
>>>>>>>>
maxThreads="150" SSLEnabled="true" scheme="https"
>>>>>>>> secure="true" clientAuth="false" sslProtocol="TLS"
>>>>>>>> /> -->
>>>>>>>>
>>>>>>>> <!-- Define an AJP 1.3 Connector on port 8009 -->
>>>>>>>> <Connector port="8009" protocol="AJP/1.3"
>>>>>>>> redirectPort="8443" />
>>>>>>>>
>>>>>>>>
>>>>>>>> <!-- An Engine represents the entry point (within
>>>>>>>> Catalina) that processes every request. The Engine
>>>>>>>> implementation for Tomcat stand alone analyzes the
>>>>>>>> HTTP headers included with the request, and passes
>>>>>>>> them on to the appropriate Host (virtual host).
>>>>>>>> Documentation at /docs/config/engine.html -->
>>>>>>>>
>>>>>>>> <!-- You should set jvmRoute to support
>>>>>>>> load-balancing via AJP ie :
>>>>>>>>
>>>>>>>> --> <Engine name="Catalina" defaultHost="localhost"
>>>>>>>> jvmRoute="TomcatNode1"> <!-- Use the LockOutRealm to
>>>>>>>> prevent attempts to guess user passwords via a
>>>>>>>> brute-force attack --> <Realm
>>>>>>>> className="org.apache.catalina.realm.LockOutRealm">
>>>>>>>> <!-- This Realm uses the UserDatabase configured in
>>>>>>>> the global JNDI resources under the key
>>>>>>>> "UserDatabase". Any edits that are performed against
>>>>>>>> this UserDatabase are immediately available for use
>>>>>>>> by the Realm. --> <Realm
>>>>>>>> className="org.apache.catalina.realm.UserDatabaseRealm"
>>>>>>>>
>>>>>>>>
resourceName="UserDatabase"/>
>>>>>>>> </Realm>
>>>>>>>>
>>>>>>>> <Host name="localhost" appBase="webapps"
>>>>>>>> unpackWARs="true" autoDeploy="true"
>>>>>>>> startStopThreads="0"> <!--For clustering, please take
>>>>>>>> a look at documentation at: /docs/cluster-howto.html
>>>>>>>> (simple how to) /docs/config/cluster.html (reference
>>>>>>>> documentation) --> <!-- <Cluster
>>>>>>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
>>>>>>>>
>>>>>>>>
- -->
>>>>>>>>
>>>>>>>> <Cluster
>>>>>>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>>>>>>>>
>>>>>>>>
channelSendOptions="4">
>>>>>>>> <Channel
>>>>>>>> className="org.apache.catalina.tribes.group.GroupChannel">
>>>>>>>>
>>>>>>>>
<Membership
>>>>>>>> className="org.apache.catalina.tribes.membership.McastService"
>>>>>>>>
>>>>>>>>
address="228.0.0.4"
>>>>>>>> port="45564" frequency="500" dropTime="9000"/>
>>>>>>>> <Receiver
>>>>>>>> className="org.apache.catalina.tribes.transport.nio.NioReceiver
"
>>>>>>>>
>>>>>>>>
address="auto"
>>>>>>>> port="4000" autoBind="100" selectorTimeout="5000"
>>>>>>>> maxThreads="6"/>
>>>>>>>>
>>>>>>>> <Sender
>>>>>>>>
>>>>>>>> className="org.apache.catalina.tribes.transport.ReplicationTran
smitter">
>>>>>>>>
>>>>>>>>
<Transport
>>>>>>>>
>>>>>>>>
>>>>>>>> className="org.apache.catalina.tribes.transport.nio.PooledParal
lelSender"/>
>>>>>>>>
>>>>>>>>
</Sender>
>>>>>>>> <Interceptor
>>>>>>>>
>>>>>>>>
>>>>>>>> className="org.apache.catalina.tribes.group.interceptors.TcpFai
lureDetector"/>
>>>>>>>>
>>>>>>>>
<Interceptor
>>>>>>>>
>>>>>>>>
>>>>>>>> className="org.apache.catalina.tribes.group.interceptors.Messag
eDispatch15Interceptor"/>
>>>>>>>>
>>>>>>>>
</Channel>
>>>>>>>>
>>>>>>>> <Valve
>>>>>>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>>>>>>
>>>>>>>>
filter=""/>
>>>>>>>> <Valve
>>>>>>>> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/
>
>>>>>>>>
>>>>>>>>
>>>>>>>>
<Deployer
>>>>>>>> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>>>>>>>>
>>>>>>>>
tempDir="D:/imaljava/TomcatNode1/tmp/war-temp/"
>>>>>>>>
>>>>>>>> deployDir="D:/imaljava/TomcatNode1/tmp/war-deploy/"
>>>>>>>>
>>>>>>>> watchDir="D:/imaljava/TomcatNode1/tmp/war-listen/"
>>>>>>>> watchEnabled="false"/>
>>>>>>>>
>>>>>>>> <ClusterListener
>>>>>>>> className="org.apache.catalina.ha.session.ClusterSessionListene
r"/>
>>>>>>>>
>>>>>>>>
</Cluster>
>>>>>>>> <!-- SingleSignOn valve, share authentication between
>>>>>>>> web applications Documentation at:
>>>>>>>> /docs/config/valve.html --> <!-- <Valve
>>>>>>>> className="org.apache.catalina.authenticator.SingleSignOn"
>>>>>>>> /> -->
>>>>>>>>
>>>>>>>> <!-- Access log processes all example. Documentation
>>>>>>>> at: /docs/config/valve.html Note: The pattern used is
>>>>>>>> equivalent to using pattern="common" --> <Valve
>>>>>>>> className="org.apache.catalina.valves.AccessLogValve"
>>>>>>>>
>>>>>>>>
directory="logs"
>>>>>>>> prefix="localhost_access_log" suffix=".txt"
>>>>>>>> pattern="%h %l %u %t "%r" %s %b" /> <Valve
>>>>>>>> className="org.apache.catalina.valves.StuckThreadDetectionValve
"
>>>>>>>>
>>>>>>>>
threshold="900" />
>>>>>>>>
>>>>>>>> </Host> </Engine> </Service> </Server>
>>>>>>>>
>>>>>>>> On Mon, Feb 6, 2017 at 6:51 PM, André Warnier
>>>>>>>> (tomcat) <a...@ice-sa.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 06.02.2017 17:45, Fady Haikal wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi, What is the host OS ? Windows Server 2012
>>>>>>>>>> What is the Tomcat version ? Apache
>>>>>>>>>> Tomcat/8.0.30
>>>>>>>>>>
>>>>>>>>>> Is this problem new ? was this working before ?
>>>>>>>>>> how long ? Since cluster implementation
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I still don't know tribes, but then my non-educated
>>>>>>>>> guess at this point would be that there is
>>>>>>>>> something wrong in your configuration. Can you
>>>>>>>>> copy/paste it here ? (remove sensible things like
>>>>>>>>> passwords, public IP addresses etc..)(but not to
>>>>>>>>> the point of making it uncheckable).
>>>>>>>>>
>>>>>>>>> Then maybe some tribes-specialist can take over ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Is there actually something listening on that
>>>>>>>>>> address/port ? Tomcat cluster
>>>>>>>>>>
>>>>>>>>>> the Port 4000 is listening and there is no
>>>>>>>>>> disconnection between 2 nodes ping and telnet are
>>>>>>>>>> OK
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 6, 2017 at 6:42 PM, André Warnier
>>>>>>>>>> (tomcat) <a...@ice-sa.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 06.02.2017 17:24, Fady Haikal wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Plz can i get some help here? This issue is
>>>>>>>>>>>> still occurring and it's filling the log file
>>>>>>>>>>>> in the Production server
>>>>>>>>>>>>
>>>>>>>>>>>> Regards, Fady
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi. If you want quick answers, you should
>>>>>>>>>>> provide more information. What is the host OS
>>>>>>>>>>> ? What is the Tomcat version ? Is this problem
>>>>>>>>>>> new ? was this working before ? how long ?
>>>>>>>>>>>
>>>>>>>>>>> I do not know tribes at all, but according to
>>>>>>>>>>> the logfile below, it seems that something is
>>>>>>>>>>> trying to "ping" the address 10.114.43.103,
>>>>>>>>>>> port 4000, and never getting a response (or at
>>>>>>>>>>> least not within 3000ms). Is there actually
>>>>>>>>>>> something listening on that address/port ? The
>>>>>>>>>>> "netstat" command (available both on Linux and
>>>>>>>>>>> Windows) can tell you. If there is something
>>>>>>>>>>> listening there, can it respond to whatever is
>>>>>>>>>>> ping-ing it ? (routing, firewall, ..)
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 6, 2017 at 8:52 AM, Fady Haikal
>>>>>>>>>>>> <fadyhai...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Guys, we are facing the below errors in
>>>>>>>>>>>>> Tomcat cluster, please advise
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 06-Feb-2017 01:14:20.718 SEVERE
>>>>>>>>>>>>> [GroupChannel-Heartbeat-1]
>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
rtbeat
>>>>>>>>>>>>>
>>>>>>>>>>>>>
Unable to send AbstractReplicatedMap.ping message
>>>>>>>>>>>>> org.apache.catalina.tribes.ChannelException:
>>>>>>>>>>>>> Operation has timed out(3000 ms.).; Faulty
>>>>>>>>>>>>> members:tcp://{10, 114, 43, 103}:4000; at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender
.sendMessage(ParallelNioSender.java:108)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.PooledParallelSen
der.sendMessage(PooledParallelSender.java:48)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.ReplicationTransmitte
r.sendMessage(ReplicationTransmitter.java:54)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMe
ssage(ChannelCoordinator.java:82)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.MessageDispa
tchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDe
tector.sendMessage(TcpFailureDetector.java:93)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
annel.java:233)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
annel.java:186)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.RpcChannel.send(RpcChanne
l.java:99)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.pin
g(AbstractReplicatedMap.java:267)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
rtbeat(AbstractReplicatedMap.java:885)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.heartbeat(Gr
oupChannel.java:161)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel$HeartbeatThr
ead.run(GroupChannel.java:697)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
06-Feb-2017 01:20:51.437 SEVERE [NioReceiver]
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.liste
n
>>>>>>>>>>>>>
>>>>>>>>>>>>>
Unable
>>>>>>>>>>>>> to process request in NioReceiver
>>>>>>>>>>>>> java.io.IOException: A non-blocking socket
>>>>>>>>>>>>> operation could not be completed
>>>>>>>>>>>>> immediately at
>>>>>>>>>>>>> sun.nio.ch.SocketDispatcher.close0(Native
>>>>>>>>>>>>> Method) at
>>>>>>>>>>>>> sun.nio.ch.SocketDispatcher.close(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.SocketChannelImpl.kill(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.select(Unknown
>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.liste
n(NioReceiver.java:272)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.run(N
ioReceiver.java:425)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
at java.lang.Thread.run(Unknown Source)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards, Fady
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -----------------------------------------------------------
- ----------
>>>>>>>>>>>>
>>>>>>>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>> users-h...@tomcat.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------
- ---------
>>>>>>>>>>>
>>>>>>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> users-h...@tomcat.apache.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------
- --------
>>>>>>>>>>
>>>>>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> users-h...@tomcat.apache.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --------------------------------------------------------------
- -------
>>>>>>>>>
>>>>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> users-h...@tomcat.apache.org
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------
- ------
>>>>>>>>
>>>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> users-h...@tomcat.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>> -----------------------------------------------------------------
- ----
>>>>>>
>>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> users-h...@tomcat.apache.org
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------
- ---
>>>>>
>>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>>> For additional commands, e-mail:
>>>>> users-h...@tomcat.apache.org
>>>>>
>>>>
>>>> -------------------------------------------------------------------
- --
>>>>
>>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>> For additional commands, e-mail:
>>>> users-h...@tomcat.apache.org
>>>>
>>>
>>>
>>> --------------------------------------------------------------------
- -
>>>
>>>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJYmfN4AAoJEBzwKT+lPKRYZuIQAI2NXtOMNcNpd88y2rXRZFCK
p3YN8OBo07UMkT+N6OgmkCrWBGRqIk9SZSUugnWP7N1n5nOFpaio/I2n/2dGOXDB
oYfPA3KVkWaMQenOAujjyodi7Q7ABUurJ07V+D0zfBtPpqE7up9/edl94pq8znuR
KGp1eEKeMlMiZAS9vm5/0ifhhMIDGfE/Y3xjqf+Kb5gHXlA5BrOrlo7+UX26uuqr
qIIaobTCKYaQvYE5HP+9FyOsXCd1oicKz9JIPAoc8A7Yj//n+5Ogtoczyu+mSzxL
X7/JSRdywbyfFo+1bOaecxmx9Bh9UgyziEh2AuXXMGwf17a58ciN8VCqtJcJC3HO
rffZtc9sqm/Tyt5tvo5PgPbpTrvsjeZXMrxUx5QuOjWlkoVOPptACB4LATVmXaoM
lHGw0OBCQgNViI5LXMs59QSdj9/J7tzN+0AakFmO0GM2fHjiFzgHTLGa5QA9yO7b
wIDpCQj3cDFig+4z95ZFHcGTmcUd9u5a6dO/4BPzFvsR/r6kfmH7qV1mwVu1TkEH
4B6EO5AwCwZJNM2asxFktswNoI4Wl2pP7ZjdxW8tIXchJco0iHn/VhnO1kPeNI8x
ws6KXUpZqPp4ylLWYKTLHg9fMc9Ffn6u6jR09o9amEIWnSlinA4R+n9VztWiBR45
h9q0x3f+aKuQp2IE0UBE
=WoKC
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
