The prefetch size was set on the network connector as we were getting messages about slow consumers across the network bridge.
As far as I can see the network bridge had not failed. The connector entries in the log are for a client subscription that will also have the topic prefetch set to 32766. I am trying to get logs from the client. The broker on the other end of the bridge uses the same configuration. On 27 November 2012 13:41, Christian Posta <christian.po...@gmail.com>wrote: > Answers to your questions: > > 1) Not sure yet > 2) Because at the moment, send fail if no space is only triggered when > producer flow control is on (at least for this case, topics) > 3) like gtully said, connections could not be shutdown if they are blocked > somehow > > I noticed in your config you explicitly set the prefetch on the network > connector to 32766. The default for network connectors is 1000 and the > default for regular topics is Short.MAX_VALUE (which is 32767). Since the > bridge doesn't have a prefetch buffer like normal clients do, setting the > prefetch to 32766 could end up flooding it. Any reason why you have it set > to 32766? > > So TopicSubscriptions should always have the broker's main memory usage. If > it has the destination's memory limit, then something went wrong. Like Gary > said, the pending message cursor's messages would be spooled to disk when > the main memory limit reaches its high water mark (70% by default).... but > that appears to not have happened in this case. > > Are there any indications that the TopicSubscription is for the network > bridge? Or maybe that the network bridge failed somehow? I see that the > dispatched count is the same as what you've set for you prefertch on the > bridge, but if anything else can point to that it might be helpful. For > example, are those port numbers on the transport connector logs for the > network bridge? > > How is the broker on the other end of the bridge configured? Same? > > > On Fri, Nov 23, 2012 at 8:36 AM, Mark Anderson <manderso...@gmail.com > >wrote: > > > I have ActiveMQ 5.6.0 configured as follows: > > > > Producer Flow Control = false > > Send Fail If No Space = true > > Memory Usage Limit = 128Mb > > Temp Usage Limit = 1Gb > > > > All my messages are non-persistent. The temp usage is configured to > handle > > spikes/slow consumers when processing messages. > > > > I continually see the following in the logs: > > > > WARN Nov 20 20:55:47 (13748874 [InactivityMonitor Async Task: > > java.util.concurrent.ThreadPoolExecutor$Worker@7ea0e15b[State = 0, empty > > queue]] org.apac > > he.activemq.broker.TransportConnection.Transport) Transport Connection > to: > > tcp://192.168.2.103:35186 failed: java.net.SocketException: Broken pipe > > INFO Nov 20 20:55:51 (13752162 [ActiveMQ Transport: tcp:/// > > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > > connection to 'tcp: > > //192.168.2.103:35166' is taking a long time to shutdown. > > INFO Nov 20 20:55:56 (13757162 [ActiveMQ Transport: tcp:/// > > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > > connection to 'tcp: > > //192.168.2.103:35166' is taking a long time to shutdown. > > INFO Nov 20 20:56:01 (13762162 [ActiveMQ Transport: tcp:/// > > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The > > connection to 'tcp: > > //192.168.2.103:35166' is taking a long time to shutdown. > > > > I'm not sure why the connection will never shutdown. > > > > I then see the following message: > > > > org.apache.activemq.broker.region.TopicSubscription) TopicSubscription: > > consumer=ID:linux-5ks2-57958-1353426643811-3:1:378:1, destinations=1, > > dispatched=32766, delivered=0, matched=0, discarded=0: Pending message > > cursor > > > > > [org.apache.activemq.broker.region.cursors.FilePendingMessageCursor@4c41cfa2 > > ] > > is full, temp usage (0%) or memory usage (211%) limit reached, blocking > > message add() pending the release of resources. > > > > This leads me to the following questions: > > > > 1) Why would the memory usage be 211% while temp usage is 0%. > > 2) The thread dump shows that send calls on producers are blocking. Why > > would they not throw exceptions when send fail if no space = true? > > 3) Would the issue with connection shutdown contribute to the memory > usage? > > > > Thanks, > > Mark > > > > > > -- > *Christian Posta* > http://www.christianposta.com/blog > twitter: @christianposta >
