So I tried that and the client appears to be sending just fine, but the producer on the other side of the network connector that expects to hear back via the temp queue is not receiving the message.
Chris On Wed, May 16, 2012 at 1:27 AM, Gary Tully <gary.tu...@gmail.com> wrote: > you need one additional bit of config, to disable the connection from > tracking temp destination advisories so that it allows the request to > go to the broker. > > factory.setWatchTopicAdvisories(false); > or ?jms.watchTopicAdvisories=false on the connection factory broker url > > On 15 May 2012 23:30, Chris Robison <chrisdrobi...@gmail.com> wrote: > > I tried that last suggestion and it appears as though > > the allowTempAutoCreationOnSend is not working. The client keep > receiving a > > "cannot send to a deleted destination" error. > > > > Chris > > > > On Mon, May 14, 2012 at 4:19 PM, Gary Tully <gary.tu...@gmail.com> > wrote: > > > >> this is expected, but the flooding impact should be minimal with a > >> small number of brokers in the network and a reasonable number of > >> connections. > >> > >> each network bridge will register interest in all temp destination > >> advisories, so that it will be in a position to receive messages for > >> those destinations. It can't know in advance which messages may be > >> routed though it, so it needs to react to each advisory message and > >> create a local temp destination. When the connection that creates the > >> temp is closes, the temp destination gets closed and each broker > >> reacts to that event via the remove temp advisory. > >> > >> Temp destinations are often pooled for serial request reply messages > >> or multiplexed using selectors, to reduce the overhead of creation and > >> propagation, in a network scenario. The other option is to use regular > >> destinations for replys > >> > >> For temp destinations, there is an alternative solution that would > >> require a little more configuration but which would reduce the > >> overhead of unnecessary temp destination propagation. > >> You can disable auto temp destination propagation through the network > >> bridge, ie: bridgeTempDestinations=false. > >> Then you need to allow temp destination creation on the fly for a > >> producer, broker.allowTempAutoCreationOnSend=true > >> To finish, you need to configure > >> broker.schedulePeriodForDestinationPurge>0 to have the on the fly > >> created temp destinations purged when they are empty and enable it for > >> temp destinations. policy entry gcInactiveDestinations=true (this is > >> important because the auto created temp destination cannot be > >> associated with a connection in any sensible way) > >> > >> There is a test that exercises these options > >> > >> > http://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/usecases/RequestReplyNoAdvisoryNetworkTest.java > >> > >> > >> On 11 May 2012 17:45, Chris Robison <chrisdrobi...@gmail.com> wrote: > >> > Another interesting observation: as a test, I connected all producers > and > >> > consumers to MSTMIP102 (the machine that doesn't have the network > >> > connector). I then started MSIPAP101 so that its network connector > could > >> > start and connect to MSTMIP102. Request request-reply works fine > between > >> > all producers and consumers, but after 1 or 2 request-replies, > MSIPAP101 > >> is > >> > flooded with the temp queue advisories--even though there are no > >> producers > >> > or consumers connected to it. > >> > > >> > On Fri, May 11, 2012 at 10:09 AM, Chris Robison < > chrisdrobi...@gmail.com > >> >wrote: > >> > > >> >> Here is the log from the latest run with the TTL down to 1. It still > >> >> floods after about 1 or 2 request-reply calls. It just seems to be > >> >> alternating between add temp queue and remove temp queue messages. > >> >> > >> >> On Fri, May 11, 2012 at 9:52 AM, Chris Robison < > chrisdrobi...@gmail.com > >> >wrote: > >> >> > >> >>> Yes, because we have a spoke and hub topology where a client in one > >> spoke > >> >>> may need to request info from a client connected to a different > spoke. > >> I > >> >>> suppose I could lower that value. > >> >>> > >> >>> > >> >>> On Fri, May 11, 2012 at 9:47 AM, Gary Tully <gary.tu...@gmail.com> > >> wrote: > >> >>> > >> >>>> Do you have a good reason to have networkTTL="5", if there is only > one > >> >>>> broker, the default value of 1 will be fine. > >> >>>> > >> >>>> On 11 May 2012 16:32, Chris Robison <chrisdrobi...@gmail.com> > wrote: > >> >>>> > If it helps, I've attached the configuration files for both > >> machines. > >> >>>> > > >> >>>> > > >> >>>> > On Fri, May 11, 2012 at 9:07 AM, Chris Robison < > >> >>>> chrisdrobi...@gmail.com> > >> >>>> > wrote: > >> >>>> >> > >> >>>> >> One thing I have noticed though is that when I restart the > broker > >> on > >> >>>> the > >> >>>> >> other end, everything starts to work again until it floods > again. > >> >>>> >> > >> >>>> >> > >> >>>> >> On Fri, May 11, 2012 at 9:05 AM, Chris Robison < > >> >>>> chrisdrobi...@gmail.com> > >> >>>> >> wrote: > >> >>>> >>> > >> >>>> >>> I'm on 5.6. And all brokers remain up and running. > >> >>>> >>> > >> >>>> >>> > >> >>>> >>> On Fri, May 11, 2012 at 8:48 AM, Gary Tully < > gary.tu...@gmail.com > >> > > >> >>>> wrote: > >> >>>> >>>> > >> >>>> >>>> what version are you on? > >> >>>> >>>> Is there any chance that the broker at the other end of the > >> network > >> >>>> >>>> bridge is shutting down? > >> >>>> >>>> > >> >>>> >>>> On 11 May 2012 14:34, Chris Robison <chrisdrobi...@gmail.com> > >> >>>> wrote: > >> >>>> >>>> > I have a network of brokers and I am using a request-reply > >> system > >> >>>> >>>> > across > >> >>>> >>>> > that network, but I've noticed that after a few > request-replys, > >> >>>> the > >> >>>> >>>> > broker > >> >>>> >>>> > network will get flooded with TempQueue advisory messages, > then > >> >>>> some > >> >>>> >>>> > of the > >> >>>> >>>> > of the brokers will be reported as slow consumers and the > >> >>>> consumers > >> >>>> >>>> > won't > >> >>>> >>>> > receive messages after that. The advisory messages seem to > be > >> >>>> repeated > >> >>>> >>>> > (e.g. add temp destination, remove temp destination) for the > >> same > >> >>>> temp > >> >>>> >>>> > destinations. Not quite sure how to handle this. Any > >> suggestions? > >> >>>> >>>> > > >> >>>> >>>> > Here are some of the errors are other things I'm seeing > >> >>>> repeatedly in > >> >>>> >>>> > logs > >> >>>> >>>> > when I enable debug. In fact, when this flood occurs, tons > of > >> log > >> >>>> gets > >> >>>> >>>> > generated all at once. > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,248 | DEBUG | Error occured while > >> processing > >> >>>> async > >> >>>> >>>> > command: DestinationInfo {commandId = 40578, > responseRequired = > >> >>>> false, > >> >>>> >>>> > connectionId = ID:MSTMIP102-59588-1336713700699-3:2, > >> destination = > >> >>>> >>>> > temp-queue://ID:POTMIP102-54337-634722930065323786-1:18:1, > >> >>>> >>>> > operationType = > >> >>>> >>>> > 0, timeout = 0, brokerPath = > >> >>>> [ID:MSSD5D101-38580-1336708634042-0:1]}, > >> >>>> >>>> > exception: java.lang.IllegalStateException: Timer already > >> >>>> cancelled. | > >> >>>> >>>> > org.apache.activemq.broker.TransportConnection.Service | > >> ActiveMQ > >> >>>> >>>> > Transport: ssl:///10.60.252.28:54493 > >> >>>> >>>> > java.lang.IllegalStateException: Timer already cancelled. > >> >>>> >>>> > at java.util.Timer.sched(Timer.java:397) > >> >>>> >>>> > at java.util.Timer.schedule(Timer.java:248) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.thread.Scheduler.schedualPeriodically(Scheduler.java:49) > >> >>>> >>>> > at > >> org.apache.activemq.broker.region.Queue.start(Queue.java:889) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:143) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:289) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:161) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.AuthorizationBroker.addDestination(AuthorizationBroker.java:81) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.AuthorizationBroker.addDestinationInfo(AuthorizationBroker.java:51) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:223) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:477) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:135) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:124) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:103) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory(DemandForwardingBridgeSupport.java:588) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:420) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport$2.onCommand(DemandForwardingBridgeSupport.java:151) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204) > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,248 | WARN | Async error occurred: > >> >>>> >>>> > java.lang.IllegalStateException: Timer already cancelled. | > >> >>>> >>>> > org.apache.activemq.broker.TransportConnection.Service | > >> ActiveMQ > >> >>>> >>>> > Transport: ssl:///10.60.252.28:54493 > >> >>>> >>>> > java.lang.IllegalStateException: Timer already cancelled. > >> >>>> >>>> > at java.util.Timer.sched(Timer.java:397) > >> >>>> >>>> > at java.util.Timer.schedule(Timer.java:248) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.thread.Scheduler.schedualPeriodically(Scheduler.java:49) > >> >>>> >>>> > at > >> org.apache.activemq.broker.region.Queue.start(Queue.java:889) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:143) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:289) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:161) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.AuthorizationBroker.addDestination(AuthorizationBroker.java:81) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.AuthorizationBroker.addDestinationInfo(AuthorizationBroker.java:51) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:223) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:477) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:135) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:124) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:103) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory(DemandForwardingBridgeSupport.java:588) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:420) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.network.DemandForwardingBridgeSupport$2.onCommand(DemandForwardingBridgeSupport.java:151) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:91) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204) > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,249 | DEBUG | mstmip102.vosh.net adding > >> >>>> >>>> > destination: > >> >>>> >>>> > topic://ActiveMQ.Advisory.Connection | > >> >>>> >>>> > org.apache.activemq.broker.region.AbstractRegion | > >> StopAsync:vm:// > >> >>>> >>>> > mstmip102.vosh.net#0-23 > >> >>>> >>>> > 2012-05-11 05:24:37,250 | WARN | Failed to remove > connection > >> >>>> >>>> > ConnectionInfo {commandId = 3, responseRequired = false, > >> >>>> connectionId > >> >>>> >>>> > = > >> >>>> >>>> > ID:msipap102-50908-1336690011745-7:1, clientId = > >> >>>> >>>> > mstmip102_msipap102.vosh.net_outbound, clientIp = > >> >>>> >>>> > vm://mstmip102.vosh.net#0, > >> >>>> >>>> > userName = mqipauser, password = *****, brokerPath = null, > >> >>>> >>>> > brokerMasterConnector = false, manageable = false, > >> clientMaster = > >> >>>> >>>> > true, > >> >>>> >>>> > faultTolerant = false, failoverReconnect = false}, reason: > >> >>>> >>>> > java.lang.IllegalStateException: Timer already cancelled. | > >> >>>> >>>> > org.apache.activemq.broker.TransportConnection.Service | > >> >>>> >>>> > StopAsync:vm:// > >> >>>> >>>> > mstmip102.vosh.net#0-23 > >> >>>> >>>> > 2012-05-11 05:24:37,250 | DEBUG | Exception detail: | > >> >>>> >>>> > org.apache.activemq.broker.TransportConnection.Service | > >> >>>> >>>> > StopAsync:vm:// > >> >>>> >>>> > mstmip102.vosh.net#0-23 > >> >>>> >>>> > java.lang.IllegalStateException: Timer already cancelled. > >> >>>> >>>> > at java.util.Timer.sched(Timer.java:397) > >> >>>> >>>> > at java.util.Timer.schedule(Timer.java:248) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.thread.Scheduler.schedualPeriodically(Scheduler.java:49) > >> >>>> >>>> > at > >> org.apache.activemq.broker.region.Topic.start(Topic.java:530) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:143) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:286) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:161) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:145) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.AuthorizationBroker.addDestination(AuthorizationBroker.java:81) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:151) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:481) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:305) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:533) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:464) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:459) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.advisory.AdvisoryBroker.removeConnection(AdvisoryBroker.java:234) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.removeConnection(BrokerFilter.java:113) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransactionBroker.removeConnection(TransactionBroker.java:351) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.removeConnection(BrokerFilter.java:113) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.security.JaasAuthenticationBroker.removeConnection(JaasAuthenticationBroker.java:95) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.BrokerFilter.removeConnection(BrokerFilter.java:113) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.MutableBrokerFilter.removeConnection(MutableBrokerFilter.java:119) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:766) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection.doStop(TransportConnection.java:1068) > >> >>>> >>>> > at > >> >>>> >>>> > > >> >>>> >>>> > > >> >>>> > >> > org.apache.activemq.broker.TransportConnection$4.run(TransportConnection.java:996) > >> >>>> >>>> > at java.lang.Thread.run(Thread.java:722) > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,252 | DEBUG | mstmip102.vosh.net adding > >> >>>> >>>> > destination: > >> >>>> >>>> > temp-queue://ID:POTMIP102-54337-634722930065323786-1:18:1 | > >> >>>> >>>> > org.apache.activemq.broker.region.AbstractRegion | ActiveMQ > >> >>>> Transport: > >> >>>> >>>> > ssl:///172.17.5.78:59658 > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,341 | DEBUG | bridging > (mstmip102.vosh.net-> > >> >>>> >>>> > msipap101.vosh.net) > >> >>>> ID:MSTMIP102-59588-1336713700699-1:1:0:0:25184, > >> >>>> >>>> > consumer: ID:MSTMIP102-59588-1336713700699-4:2:1:1, > destination > >> >>>> >>>> > topic://ActiveMQ.Advisory.TempQueue, brokerPath: > >> >>>> >>>> > [ID:MSTMIP102-59588-1336713700699-0:1], message: > >> ActiveMQMessage > >> >>>> >>>> > {commandId > >> >>>> >>>> > = 0, responseRequired = false, messageId = > >> >>>> >>>> > ID:MSTMIP102-59588-1336713700699-1:1:0:0:25184, > >> >>>> originalDestination = > >> >>>> >>>> > null, > >> >>>> >>>> > originalTransactionId = null, producerId = > >> >>>> >>>> > ID:MSTMIP102-59588-1336713700699-4:1:1:1, destination = > >> >>>> >>>> > topic://ActiveMQ.Advisory.TempQueue, transactionId = null, > >> >>>> expiration > >> >>>> >>>> > = 0, > >> >>>> >>>> > timestamp = 0, arrival = 0, brokerInTime = 1336713871588, > >> >>>> >>>> > brokerOutTime = > >> >>>> >>>> > 1336713877341, correlationId = null, replyTo = null, > >> persistent = > >> >>>> >>>> > false, > >> >>>> >>>> > type = Advisory, priority = 0, groupID = null, > groupSequence = > >> 0, > >> >>>> >>>> > targetConsumerId = null, compressed = false, userID = null, > >> >>>> content = > >> >>>> >>>> > null, > >> >>>> >>>> > marshalledProperties = > >> >>>> org.apache.activemq.util.ByteSequence@567edec8, > >> >>>> >>>> > dataStructure = DestinationInfo {commandId = 0, > >> responseRequired = > >> >>>> >>>> > false, > >> >>>> >>>> > connectionId = ID:MSTMIP102-59588-1336713700699-3:2, > >> destination = > >> >>>> >>>> > temp-queue://ID:POTMIP102-54337-634722930065323786-1:18:1, > >> >>>> >>>> > operationType = > >> >>>> >>>> > 0, timeout = 0, brokerPath = null}, redeliveryCounter = 0, > >> size = > >> >>>> >>>> > 1024, > >> >>>> >>>> > properties = {originBrokerURL=ssl://MSTMIP102:61616, > >> >>>> >>>> > originBrokerId=ID:MSTMIP102-59588-1336713700699-0:1, > >> >>>> originBrokerName= > >> >>>> >>>> > mstmip102.vosh.net}, readOnlyProperties = false, > readOnlyBody > >> = > >> >>>> false, > >> >>>> >>>> > droppable = false} | > >> >>>> >>>> > org.apache.activemq.network.DemandForwardingBridgeSupport | > >> >>>> ActiveMQ > >> >>>> >>>> > Connection Dispatcher: vm://mstmip102.vosh.net#4 > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 05:24:37,341 | DEBUG | bridging > (mstmip102.vosh.net-> > >> >>>> >>>> > msipap101.vosh.net) > >> >>>> ID:MSTMIP102-59588-1336713700699-1:1:0:0:25185, > >> >>>> >>>> > consumer: ID:MSTMIP102-59588-1336713700699-4:2:1:1, > destination > >> >>>> >>>> > topic://ActiveMQ.Advisory.TempQueue, brokerPath: > >> >>>> >>>> > [ID:MSTMIP102-59588-1336713700699-0:1], message: > >> ActiveMQMessage > >> >>>> >>>> > {commandId > >> >>>> >>>> > = 0, responseRequired = false, messageId = > >> >>>> >>>> > ID:MSTMIP102-59588-1336713700699-1:1:0:0:25185, > >> >>>> originalDestination = > >> >>>> >>>> > null, > >> >>>> >>>> > originalTransactionId = null, producerId = > >> >>>> >>>> > ID:MSTMIP102-59588-1336713700699-4:1:1:1, destination = > >> >>>> >>>> > topic://ActiveMQ.Advisory.TempQueue, transactionId = null, > >> >>>> expiration > >> >>>> >>>> > = 0, > >> >>>> >>>> > timestamp = 0, arrival = 0, brokerInTime = 1336713871588, > >> >>>> >>>> > brokerOutTime = > >> >>>> >>>> > 1336713877341, correlationId = null, replyTo = null, > >> persistent = > >> >>>> >>>> > false, > >> >>>> >>>> > type = Advisory, priority = 0, groupID = null, > groupSequence = > >> 0, > >> >>>> >>>> > targetConsumerId = null, compressed = false, userID = null, > >> >>>> content = > >> >>>> >>>> > null, > >> >>>> >>>> > marshalledProperties = > >> >>>> org.apache.activemq.util.ByteSequence@235448d5, > >> >>>> >>>> > dataStructure = DestinationInfo {commandId = 0, > >> responseRequired = > >> >>>> >>>> > false, > >> >>>> >>>> > connectionId = ID:MSTMIP102-59588-1336713700699-3:2, > >> destination = > >> >>>> >>>> > temp-queue://ID:POTMIP102-54337-634722930065323786-1:18:1, > >> >>>> >>>> > operationType = > >> >>>> >>>> > 1, timeout = 0, brokerPath = null}, redeliveryCounter = 0, > >> size = > >> >>>> >>>> > 1024, > >> >>>> >>>> > properties = {originBrokerURL=ssl://MSTMIP102:61616, > >> >>>> >>>> > originBrokerId=ID:MSTMIP102-59588-1336713700699-0:1, > >> >>>> originBrokerName= > >> >>>> >>>> > mstmip102.vosh.net}, readOnlyProperties = false, > readOnlyBody > >> = > >> >>>> false, > >> >>>> >>>> > droppable = false} | > >> >>>> >>>> > org.apache.activemq.network.DemandForwardingBridgeSupport | > >> >>>> ActiveMQ > >> >>>> >>>> > Connection Dispatcher: vm://mstmip102.vosh.net#4 > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 13:29:40,053 | WARN | TopicSubscription: > >> >>>> >>>> > consumer=ID:MSTMIP102-32947-1336741838889-4:2:1:1, > >> destinations=7, > >> >>>> >>>> > dispatched=1000, delivered=7723, matched=1001, discarded=0: > has > >> >>>> twice > >> >>>> >>>> > its > >> >>>> >>>> > prefetch limit pending, without an ack; it appears to be > slow | > >> >>>> >>>> > org.apache.activemq.broker.region.TopicSubscription | > ActiveMQ > >> >>>> >>>> > Transport: > >> >>>> >>>> > ssl:///10.60.252.28:54614 > >> >>>> >>>> > > >> >>>> >>>> > 2012-05-11 13:29:42,540 | WARN | TopicSubscription: > >> >>>> >>>> > consumer=ID:MSTMIP102-32947-1336741838889-3:2:1:1, > >> destinations=6, > >> >>>> >>>> > dispatched=1000, delivered=9922, matched=1001, discarded=0: > has > >> >>>> twice > >> >>>> >>>> > its > >> >>>> >>>> > prefetch limit pending, without an ack; it appears to be > slow | > >> >>>> >>>> > org.apache.activemq.broker.region.TopicSubscription | > ActiveMQ > >> >>>> >>>> > Transport: > >> >>>> >>>> > ssl:///10.60.252.28:54614 > >> >>>> >>>> > >> >>>> >>>> > >> >>>> >>>> > >> >>>> >>>> -- > >> >>>> >>>> http://fusesource.com > >> >>>> >>>> http://blog.garytully.com > >> >>>> >>> > >> >>>> >>> > >> >>>> >> > >> >>>> > > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> http://fusesource.com > >> >>>> http://blog.garytully.com > >> >>>> > >> >>> > >> >>> > >> >> > >> > >> > >> > >> -- > >> http://fusesource.com > >> http://blog.garytully.com > >> > > > > -- > http://fusesource.com > http://blog.garytully.com >
