Some followup info: I've been playing with various settings today to no
avail. I did find one interesting piece of information however: if I "kill"
the master using the "stop" command (graceful shutdown) instead of using
"kill -9", the system behaves as I expect!
Below is the log output from my client. You can see the connection to
master1 (port 61611) is lost, but the remaining 7 messages are delivered as
expected (by master2). Messages 1-3 are delivered by master1, then you can
see the reconnect to master2. Those first 3 messages are then suppressed as
dups (yay), and the remaining msgs 4-10 are then delivered!
This seems like interesting information indeed. I was using "kill -9" as my
simulation of a drastic process failure of the broker. Is activemq
clustering usually robust when a broker is rudely "kill -9"ed?
master1 port = 61611
master2 port = 61612
slave1 port = 61613
slave2 port = 61614
0 [Notifier-MulticastDiscoveryAgent-listener:unconnected] INFO
org.apache.activemq.transport.discovery.DiscoveryTransport - Adding new
broker connection URL: tcp://brokerhost:61611
3 [Notifier-MulticastDiscoveryAgent-listener:unconnected] INFO
org.apache.activemq.transport.discovery.DiscoveryTransport - Adding new
broker connection URL: tcp://brokerhost:61612
474 [ActiveMQ Task-1] INFO
org.apache.activemq.transport.failover.FailoverTransport - Successfully
connected to tcp://brokerhost:61611
573 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
DURABLY Listening for messages on broker
discovery:(multicast://default)?maxReconnectAttempts=10
27855 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #1 (in set 53) received
28856 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#1 is done getting processed (in set 53)
28856 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #2 (in set 53) received
29857 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#2 is done getting processed (in set 53)
29858 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #3 (in set 53) received
30624 [ActiveMQ Transport: tcp://brokerhost/172.10.10.10:61611@61968] WARN
org.apache.activemq.transport.failover.FailoverTransport - Transport
(tcp://brokerhost/172.10.10.10:61611@61968) failed, attempting to
automatically reconnect
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:258)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:744)
30657 [ActiveMQ Task-3] INFO
org.apache.activemq.transport.failover.FailoverTransport - Successfully
reconnected to tcp://brokerhost:61612
30704 [ActiveMQ Session Task-1] WARN
org.apache.activemq.ActiveMQMessageConsumer -
ID:brokerhost-44480-1437091458826-1:1:1:1 suppressing duplicate delivery on
connection, poison acking: MessageDispatch {commandId = 0, responseRequired
= false, consumerId = ID:brokerhost-44480-1437091458826-1:1:1:1, destination
= topic://failover, message = ActiveMQObjectMessage {commandId = 5,
responseRequired = true, messageId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1:1, originalDestination =
null, originalTransactionId = null, producerId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1, destination =
topic://failover, transactionId = null, expiration = 0, timestamp =
1437091474029, arrival = 0, brokerInTime = 1437091487583, brokerOutTime =
1437091490519, correlationId = null, replyTo = null, persistent = true, type
= null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId =
null, compressed = false, userID = null, content = null,
marshalledProperties = org.apache.activemq.util.ByteSequence@144e3ae,
dataStructure = null, redeliveryCounter = 0, size = 0, properties =
{Set-Id=53, Sequence-Id=1}, readOnlyProperties = true, readOnlyBody = true,
droppable = false, jmsXGroupFirstForConsumer = false}, redeliveryCounter =
0}
30707 [ActiveMQ Session Task-1] WARN
org.apache.activemq.ActiveMQMessageConsumer -
ID:brokerhost-44480-1437091458826-1:1:1:1 suppressing duplicate delivery on
connection, poison acking: MessageDispatch {commandId = 0, responseRequired
= false, consumerId = ID:brokerhost-44480-1437091458826-1:1:1:1, destination
= topic://failover, message = ActiveMQObjectMessage {commandId = 6,
responseRequired = true, messageId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1:2, originalDestination =
null, originalTransactionId = null, producerId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1, destination =
topic://failover, transactionId = null, expiration = 0, timestamp =
1437091474076, arrival = 0, brokerInTime = 1437091487626, brokerOutTime =
1437091490522, correlationId = null, replyTo = null, persistent = true, type
= null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId =
null, compressed = false, userID = null, content = null,
marshalledProperties = org.apache.activemq.util.ByteSequence@1ea319b,
dataStructure = null, redeliveryCounter = 0, size = 0, properties =
{Set-Id=53, Sequence-Id=2}, readOnlyProperties = true, readOnlyBody = true,
droppable = false, jmsXGroupFirstForConsumer = false}, redeliveryCounter =
0}
30709 [ActiveMQ Session Task-1] WARN
org.apache.activemq.ActiveMQMessageConsumer -
ID:brokerhost-44480-1437091458826-1:1:1:1 suppressing duplicate delivery on
connection, poison acking: MessageDispatch {commandId = 0, responseRequired
= false, consumerId = ID:brokerhost-44480-1437091458826-1:1:1:1, destination
= topic://failover, message = ActiveMQObjectMessage {commandId = 7,
responseRequired = true, messageId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1:3, originalDestination =
null, originalTransactionId = null, producerId =
ID:jlindwall-VirtualBox-35463-1437091473626-1:1:1:1, destination =
topic://failover, transactionId = null, expiration = 0, timestamp =
1437091474085, arrival = 0, brokerInTime = 1437091487636, brokerOutTime =
1437091490523, correlationId = null, replyTo = null, persistent = true, type
= null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId =
null, compressed = false, userID = null, content = null,
marshalledProperties = org.apache.activemq.util.ByteSequence@5ebd39,
dataStructure = null, redeliveryCounter = 0, size = 0, properties =
{Set-Id=53, Sequence-Id=3}, readOnlyProperties = true, readOnlyBody = true,
droppable = false, jmsXGroupFirstForConsumer = false}, redeliveryCounter =
0}
30858 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#3 is done getting processed (in set 53)
30859 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #4 (in set 53) received
31860 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#4 is done getting processed (in set 53)
31861 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #5 (in set 53) received
32861 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#5 is done getting processed (in set 53)
32862 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #6 (in set 53) received
33862 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#6 is done getting processed (in set 53)
33863 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #7 (in set 53) received
34864 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#7 is done getting processed (in set 53)
34865 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #8 (in set 53) received
35865 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#8 is done getting processed (in set 53)
35866 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #9 (in set 53) received
36866 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#9 is done getting processed (in set 53)
36867 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
Sequence #10 (in set 53) received
37815 [Notifier-MulticastDiscoveryAgent-listener:unconnected] INFO
org.apache.activemq.transport.discovery.DiscoveryTransport - Adding new
broker connection URL: tcp://brokerhost:61613
37868 [main] INFO
com.xifin.jms.tools.activemq.ActiveMQFailOverDurableMessageListener -
#10 is done getting processed (in set 53)
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Cluster-Slave-not-forwarding-messages-tp4699365p4699412.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
