Answers to your questions: 1) Not sure yet 2) Because at the moment, send fail if no space is only triggered when producer flow control is on (at least for this case, topics) 3) like gtully said, connections could not be shutdown if they are blocked somehow
I noticed in your config you explicitly set the prefetch on the network connector to 32766. The default for network connectors is 1000 and the default for regular topics is Short.MAX_VALUE (which is 32767). Since the bridge doesn't have a prefetch buffer like normal clients do, setting the prefetch to 32766 could end up flooding it. Any reason why you have it set to 32766? So TopicSubscriptions should always have the broker's main memory usage. If it has the destination's memory limit, then something went wrong. Like Gary said, the pending message cursor's messages would be spooled to disk when the main memory limit reaches its high water mark (70% by default).... but that appears to not have happened in this case. Are there any indications that the TopicSubscription is for the network bridge? Or maybe that the network bridge failed somehow? I see that the dispatched count is the same as what you've set for you prefertch on the bridge, but if anything else can point to that it might be helpful. For example, are those port numbers on the transport connector logs for the network bridge? How is the broker on the other end of the bridge configured? Same? On Fri, Nov 23, 2012 at 8:36 AM, Mark Anderson <manderso...@gmail.com>wrote: > I have ActiveMQ 5.6.0 configured as follows: > > Producer Flow Control = false > Send Fail If No Space = true > Memory Usage Limit = 128Mb > Temp Usage Limit = 1Gb > > All my messages are non-persistent. The temp usage is configured to handle > spikes/slow consumers when processing messages. > > I continually see the following in the logs: > > WARN Nov 20 20:55:47 (13748874 [InactivityMonitor Async Task: > java.util.concurrent.ThreadPoolExecutor$Worker@7ea0e15b[State = 0, empty > queue]] org.apac > he.activemq.broker.TransportConnection.Transport) Transport Connection to: > tcp://192.168.2.103:35186 failed: java.net.SocketException: Broken pipe > INFO Nov 20 20:55:51 (13752162 [ActiveMQ Transport: tcp:/// > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > connection to 'tcp: > //192.168.2.103:35166' is taking a long time to shutdown. > INFO Nov 20 20:55:56 (13757162 [ActiveMQ Transport: tcp:/// > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > connection to 'tcp: > //192.168.2.103:35166' is taking a long time to shutdown. > INFO Nov 20 20:56:01 (13762162 [ActiveMQ Transport: tcp:/// > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > connection to 'tcp: > //192.168.2.103:35166' is taking a long time to shutdown. > > I'm not sure why the connection will never shutdown. > > I then see the following message: > > org.apache.activemq.broker.region.TopicSubscription) TopicSubscription: > consumer=ID:linux-5ks2-57958-1353426643811-3:1:378:1, destinations=1, > dispatched=32766, delivered=0, matched=0, discarded=0: Pending message > cursor > > [org.apache.activemq.broker.region.cursors.FilePendingMessageCursor@4c41cfa2 > ] > is full, temp usage (0%) or memory usage (211%) limit reached, blocking > message add() pending the release of resources. > > This leads me to the following questions: > > 1) Why would the memory usage be 211% while temp usage is 0%. > 2) The thread dump shows that send calls on producers are blocking. Why > would they not throw exceptions when send fail if no space = true? > 3) Would the issue with connection shutdown contribute to the memory usage? > > Thanks, > Mark > -- *Christian Posta* http://www.christianposta.com/blog twitter: @christianposta
