Rob, So, I would find this under the 5.2 SNAPSHOT tomorrow morning (USA central standard time)?
Kevin rajdavies wrote: > > yep - being able to reproduce - this is now fixed in trunk - but too > late for the 5.1 release > > thanks, > > Rob > > On 16 Apr 2008, at 13:54, yaussy wrote: > >> >> Rob, >> >> While I don't yet have a solid / consistent test case for you, our >> application, using a particular test scenario the app guys have, can >> reproduce the problem every time. If you have anything you want me >> to try >> out, let me know. >> >> I'm still working with my test code to get something more >> reproduceable, >> while adding some debug code to IndexManager. >> >> Kevin >> >> >> yaussy wrote: >>> >>> Still working on a consistent test case. But, I've run into >>> something >>> along the way, and I'm not sure if it's related to the original >>> exception >>> I've logged here. I was able to get the problem to happen when I >>> expanded >>> my test case to a publisher of 6 topics, at 100+/sec/topic, and a >>> consumer >>> that consumes all those topics. I have this configured to use a >>> connection for each topic, for both the publisher and the >>> consumer. I've >>> only been able to get the original exception to happen once, but >>> I've more >>> consistently hit a problem wherein no exception is thrown, but the >>> broker >>> just stops forwarding events for one of the topics. After this >>> point, the >>> publisher continues, but all events for this one topic are queued >>> to disk. >>> >>> I'm trying to see if I can reduce this test case down right now. >>> But, >>> maybe you could give this a go with the supplied consumer / >>> supplier test >>> code? >>> >>> Kevin >>> >>> >>> rajdavies wrote: >>>> >>>> ok - thx for checking! - let me know when you have a test case >>>> >>>> On 15 Apr 2008, at 15:06, yaussy wrote: >>>> >>>>> >>>>> Bad news - problem still happens with "5.2" snapshot dated 4/13. I >>>>> did not >>>>> see anymore 5.1 snapshots, but saw the 5.2 directory. The archive >>>>> still has >>>>> 5.1 for the path names, so I figured this is what I should take. >>>>> >>>>> I'm still working with it and am trying to see if I can get any >>>>> more >>>>> debug >>>>> information. I have not been able to reproduce it with a test >>>>> program yet. >>>>> >>>>> >>>>> >>>>> yaussy wrote: >>>>>> >>>>>> Rob, >>>>>> >>>>>> This is good news - I will try using last nights snapshot. >>>>>> >>>>>> I started out using AMQMessageStore, but I've never had good >>>>>> performance >>>>>> luck with AMQs Journal - it always seems to have this frequent, >>>>>> high CPU >>>>>> cost as it checkpoints. Our Durable rate requirements are fairly >>>>>> significant, such as a few hundred events per second to a >>>>>> handful of >>>>>> consumers, on one of our clusters. It has not performed well, >>>>>> whereas >>>>>> Kaha by itself is excellent. >>>>>> >>>>>> Kevin >>>>>> >>>>>> >>>>>> rajdavies wrote: >>>>>>> >>>>>>> >>>>>>> On 14 Apr 2008, at 14:07, yaussy wrote: >>>>>>> >>>>>>>> >>>>>>>> I have not been able to reproduce this problem outside of our >>>>>>>> application, >>>>>>>> but I'm still trying. >>>>>>>> >>>>>>>> Anyway, I'm using regular Kaha persistence (not >>>>>>>> AMQMessageStore), >>>>>>>> and during >>>>>>>> a Durable Topic test, I'm eventually getting the following >>>>>>>> exception >>>>>>>> in the >>>>>>>> AMQBroker, after which the broker stops forwarding events to the >>>>>>>> consumer. >>>>>>>> >>>>>>>> Anyone seen this? >>>>>>>> >>>>>>>> >>>>>>>> < >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq.broker.region.cursors.AbstractStoreCursor.fillBatch> >>>>>>>> <Thread[ActiveMQ Transport: tcp:///127.0.0.1:43595,4,main]> >>>>>>>> Failed >>>>>>>> to fill >>>>>>>> batch >>>>>>>> Stack Trace follows: >>>>>>>> java.lang.RuntimeException: Failed to get next index from >>>>>>>> IndexManager:(index-topic-subs) for offset=7681977, key=(2, >>>>>>>> 20080006, 47), >>>>>>>> value=(2, 20080058, 165), previousItem=7681212, nextItem=7682895 >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .kaha >>>>>>>> .impl >>>>>>>> .index >>>>>>>> .DiskIndexLinkedList.getNextEntry(DiskIndexLinkedList.java: >>>>>>>> 267) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .kaha >>>>>>>> .impl.container.MapContainerImpl.getNext(MapContainerImpl.java: >>>>>>>> 449) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .store >>>>>>>> .kahadaptor >>>>>>>> .TopicSubContainer.getNextEntry(TopicSubContainer.java: >>>>>>>> 95) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .store >>>>>>>> .kahadaptor >>>>>>>> .KahaTopicMessageStore >>>>>>>> .recoverNextMessages(KahaTopicMessageStore.java:165) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .store >>>>>>>> .ProxyTopicMessageStore >>>>>>>> .recoverNextMessages(ProxyTopicMessageStore.java:97) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .cursors.TopicStorePrefetch.doFillBatch(TopicStorePrefetch.java: >>>>>>>> 107) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java: >>>>>>>> 188) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java: >>>>>>>> 104) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .cursors >>>>>>>> .StoreDurableSubscriberCursor >>>>>>>> .hasNext(StoreDurableSubscriberCursor.java:210 >>>>>>>> ) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .PrefetchSubscription.dispatchPending(PrefetchSubscription.java: >>>>>>>> 479) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .region >>>>>>>> .PrefetchSubscription.acknowledge(PrefetchSubscription.java: >>>>>>>> 357) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker.region.AbstractRegion.acknowledge(AbstractRegion.java: >>>>>>>> 349) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker.region.RegionBroker.acknowledge(RegionBroker.java: >>>>>>>> 474) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker.TransactionBroker.acknowledge(TransactionBroker.java: >>>>>>>> 194) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .MutableBrokerFilter.acknowledge(MutableBrokerFilter.java: >>>>>>>> 84) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker >>>>>>>> .TransportConnection.processMessageAck(TransportConnection.java: >>>>>>>> 444) >>>>>>>> at >>>>>>>> org.apache.activemq.command.MessageAck.visit(MessageAck.java: >>>>>>>> 196) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .broker.TransportConnection.service(TransportConnection.java: >>>>>>>> 293) >>>>>>>> at >>>>>>>> org.apache.activemq.broker.TransportConnection >>>>>>>> $1.onCommand(TransportConnection.java:181) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .transport.TransportFilter.onCommand(TransportFilter.java:68) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .transport >>>>>>>> .WireFormatNegotiator.onCommand(WireFormatNegotiator.java: >>>>>>>> 143) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .transport.InactivityMonitor.onCommand(InactivityMonitor.java: >>>>>>>> 206) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .transport.TransportSupport.doConsume(TransportSupport.java: >>>>>>>> 84) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java: >>>>>>>> 196) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq.transport.tcp.TcpTransport.run(TcpTransport.java: >>>>>>>> 183) >>>>>>>> at java.lang.Thread.run(Thread.java:619) >>>>>>>> Caused by: java.io.EOFException >>>>>>>> at >>>>>>>> java.io.RandomAccessFile.readFully(RandomAccessFile.java: >>>>>>>> 383) >>>>>>>> at >>>>>>>> java.io.RandomAccessFile.readFully(RandomAccessFile.java: >>>>>>>> 361) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .kaha >>>>>>>> .impl.index.StoreIndexReader.readItem(StoreIndexReader.java: >>>>>>>> 46) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .kaha.impl.index.IndexManager.getIndex(IndexManager.java: >>>>>>>> 66) >>>>>>>> at >>>>>>>> org >>>>>>>> .apache >>>>>>>> .activemq >>>>>>>> .kaha >>>>>>>> .impl >>>>>>>> .index >>>>>>>> .DiskIndexLinkedList.getNextEntry(DiskIndexLinkedList.java: >>>>>>>> 265) >>>>>>>> ... 27 more >>>>>>>> >>>>>>>> -- >>>>>>>> View this message in context: >>>>>>>> http://www.nabble.com/Kaha-persistence-issue-w-5.1-%28upto-4-8-SNAPSHOT%29-tp16677512s2354p16677512.html >>>>>>>> Sent from the ActiveMQ - User mailing list archive at >>>>>>>> Nabble.com. >>>>>>>> >>>>>>> >>>>>>> There was a bug fixed in this area last Friday - would be >>>>>>> interested >>>>>>> if this is still a problem for you with a later version? >>>>>>> If it is - Ill digg a little deeper. >>>>>>> >>>>>>> btw - why are you using Kaha instead of AMQStore ? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> cheers, >>>>>>> >>>>>>> Rob >>>>>>> >>>>>>> http://open.iona.com/ -Enterprise Open Integration >>>>>>> http://rajdavies.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://www.nabble.com/Kaha-persistence-issue-w-5.1-%28upto-4-8-SNAPSHOT%29-tp16677512s2354p16700798.html >>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >>>>> >>>> >>>> >>>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Kaha-persistence-issue-w-5.1-%28upto-4-8-SNAPSHOT%29-tp16677512s2354p16721669.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> > > > -- View this message in context: http://www.nabble.com/Kaha-persistence-issue-w-5.1-%28upto-4-8-SNAPSHOT%29-tp16677512s2354p16725769.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
