I doubt I can reproduce it as a unit test: What I see happens (sometimes, sometimes not) because I sigkill the process. If I replace the kill with an orderly shutdown, everything works out ok.
On Tue, Oct 23, 2012 at 3:38 PM, Gary Tully <gary.tu...@gmail.com> wrote: > do you think you could come up with a unit test that shows growth. So > something that uses mkahadb and produces or consumed in a transaction > from destinations that span the stores. > > You could configure the size of journal data files to 16k or > something, such that any issue shows up very quickly. > > A test case like that would help clarify the use case and if it can > reproduce, would be a great start to getting a quick resolution. > > There is a test that does transacted send received across stores, but > does not check for cleanup. It may be easy to extend it in that way. > > in activemq-core > org.apache.activemq.store.StorePerDestinationTest#testTransactedSendReceiveAcrossStores > > > On 23 October 2012 14:20, Gilles Harloux <gilles.harl...@gmail.com> wrote: >> I found that link while you were answering, it seems <g>. I also found >> https://issues.apache.org/jira/browse/AMQ-3866 and it looks a lot like >> what I experience. >> >> I tried to delete the index, start a broker and debug as explained in >> that ticket. Breakpointing at MessageDatabase#process, line 904 >> (version 5.7.0) tells me it seems there is XA in the mix (data is a >> KahaAddMessageCommand instance, with >> data.f_transactionInfo.f_localTransactionId == null and >> data.f_transactionInfo.f_xaTransactionId != null). I guess it's due to >> the use of mKahaDB, where I am transacting across both stores. >> >> In that ticket, you asked for the relevant data file. I can share mine >> if there's any interest -- I just have to find a convenient way to >> provide you with a 32 Mb file. >> >> On Tue, Oct 23, 2012 at 12:24 PM, Gary Tully <gary.tu...@gmail.com> wrote: >>> have a read of >>> http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html >>> >>> The logging referenced there will show you what destinations are >>> holding on to references to the journal data files. >>> >>> w.r.t the usage %, the journal size increases in chunks of data file >>> size, so a new journal data file can push the usage over the limit. >>> >>> On 23 October 2012 09:15, Gilles Harloux <gilles.harl...@gmail.com> wrote: >>>> Hi, >>>> >>>> I have a system embedding a broker with KahaDB as a store. I am trying >>>> to get a feel about disaster recovery behavior. So what I am basically >>>> doing is randomly kill & restart the process. I see a condition (after >>>> killing & restarting multiple times) where the messages get consumed, >>>> but kahaDB journal files (db-*.log) don't get reclaimed. As I set up a >>>> storeUsage limit, it ends up blocking the system. >>>> >>>> I say the messages get consumed, because I can see my application >>>> logging what messages it gets to handle; Also, JMX tells me that the >>>> broker's TotalMesageCount is zero, while the StorePercentUsage is >>>> above 100 (depending on parameters such as message size & rate, I saw >>>> anything from 112 to 293 percent usage). >>>> >>>> So, two questions: >>>> - How is it possible (for any reasons) to get a usage percentage >>>> above 100? (In other situations, I saw it happen with memory too). >>>> - Why is it I can't get kahaDB to reclaim seemingly unused journal files? >>>> >>>> TIA, >>> >>> >>> >>> -- >>> http://redhat.com >>> http://blog.garytully.com > > > > -- > http://redhat.com > http://blog.garytully.com
