Hi there! We are having some issues and need a small sanity check :')
We are running a network of brokers (multicast) with three nodes. Consumers and producers can connect to any of three nodes in the network/cluster. From time to time, we see that messages start to expire for one or more queues. When this happens, it is usually the case that the node where the message expiry occurs, does not have any directly connected consumers. It is as the messages are stuck on the node where the expiry occurs. We are trying to figure out what might be wrong with our config, when removing broker3 everything seems stable and working. Without any "stuck" messages expiring. The nodes are strongly coupled with a 10G link, so we have no issues with passing along messages between nodes as need be This is our current config (sans auth and queues/topics): [cid:3e730e77-7bc3-4790-9971-454add40a342] <beans xmlns="http://www.springframework.org/schema/beans"; xmlns:amq="http://activemq.apache.org/schema/core"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd";> <!-- Allows us to use system properties as variables in this configuration file --> <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <property name="locations"> <value>file:${activemq.conf}/credentials.properties</value> </property> </bean> <broker xmlns="http://activemq.apache.org/schema/core"; brokerName="<%= @fqdn %>" dataDirectory="${activemq.data}" useJmx="true" persistent="false" > <plugins> <discardingDLQBrokerPlugin dropAll="true" /> <simpleAuthenticationPlugin> <users> <authenticationUser username="example" password="example" groups="producers,consumers"/> </users> </simpleAuthenticationPlugin> <authorizationPlugin> <map> <authorizationMap> <authorizationEntries> <authorizationEntry queue="Service.>" read="consumers" write="producers" admin="admins"/> <authorizationEntry topic="Service.>" read="consumers" write="producers" admin="admins"/> <authorizationEntry topic="ActiveMQ.Advisory.>" read="consumers,producers,guests,users" write="consumers,producers,guests,users" admin="consumers,producers,guests,users,admins"/> </authorizationEntries> </authorizationMap> </map> </authorizationPlugin> </plugins> <destinationPolicy> <policyMap> <policyEntries> <!-- Set the following policy on all queues using the '>' wildcard --> <policyEntry queue=">" prioritizedMessages="true"/> <policyEntry queue=">"> <!-- Tell the dead letter strategy not to process expired messages so that they will just be discarded instead of being sent to the DLQ --> <deadLetterStrategy> <sharedDeadLetterStrategy processExpired="false" /> </deadLetterStrategy> </policyEntry> <policyEntry topic="Service.>"> <networkBridgeFilterFactory> <conditionalNetworkBridgeFilterFactory replayWhenNoConsumers="true"/> </networkBridgeFilterFactory> </policyEntry> </policyEntries> </policyMap> </destinationPolicy> <destinations> <queue physicalName="Service.Example" /> <topic physicalName="Service.Example.ALL" /> </destinations> <persistenceAdapter> <memoryPersistenceAdapter/> </persistenceAdapter> <transportConnectors> <!-- Some Puppet template marup, @fqdn is resolved to the hostname of the node --> <transportConnector name="listener" uri="tcp://<%= @fqdn %>:4001?maximumConnections=1000&wireformat.maxFrameSize=104857600" rebalanceClusterClients="true" updateClusterClients="true" updateClusterClientsOnRemove="true"/> <transportConnector name="multicast" uri="tcp://<%= @fqdn %>:4200" discoveryUri="multicast://default"/> </transportConnectors> <networkConnectors> <!-- Only Message TTL? Might need to set the other TTL too? --> <networkConnector uri="multicast://default" userName="broker" password="password" decreaseNetworkConsumerPriority="false" messageTTL="5" suppressDuplicateQueueSubscriptions="false" conduitSubscriptions="false"/> </networkConnectors> <shutdownHooks> <bean xmlns="http://www.springframework.org/schema/beans"; class="org.apache.activemq.hooks.SpringContextHook" /> </shutdownHooks> </broker> <import resource="jetty.xml"/> </beans> -Nikolas Papaioannou
