Timothy,
Thank you for the quick response!
I had read that, but didn't see anything useful in it. Having re-read it,
I realize that my DLQs (which contain 100k messages) were a likely suspect
in keeping the log files "live." I cleared the DLQs, and ActiveMQ
diligently deleted most of the log files a couple minutes later:
$ cd apache-activemq-5.8.0/data/kahadb/
$ ls -lht | head
$ ls -lht | head
total 4.9G
-rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 18:00 db-1564.log
-rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 18:00 db.data
-rw-rw-r-- 1 ubuntu ubuntu 2.6M Dec 13 18:00 db.redo
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 17:44 db-1563.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 16:14 db-1562.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
Thank you for the hint, Timothy!
Andy Boothe
Director of Data Engineering, Analytics
WCG
60 Francisco St, San Francisco, 94133
direct 512-961-3993 cell 979-574-1089
twitter @sigpwned <http://twitter.com/sigpwned> linkedin aboothe
<http://www.linkedin.com/in/aboothe>
github sigpwned <https://github.com/sigpwned> stack overflow sigpwned
<http://stackoverflow.com/users/2103602/sigpwned>
€ € € € € € € Go. Ahead.
ŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠ
On 12/13/13 10:59 AM, "Timothy Bish" <tabish...@gmail.com> wrote:
>On 12/13/2013 11:27 AM, Andy Boothe [WCG] wrote:
>> Hello,
>>
>> I have an ActiveMQ instance that has been running nicely for a couple
>> of months, but doesn't seem to clean up its KahaDB logs like I expect
>> it to. (More specifically, I expect to have like 1 log file, but in
>> reality I have ~1500 log files with 49G of data.)
>>
>> ActiveMQ has processed about 12M messages over the past couple of
>> months, and has roughly 106K messages currently pending (in DLQs):
>>
>>
>> Given the number of messages currently pending, I would not expect the
>> KahaDB logs to be very big. In reality, they're about 49G:
>>
>> $ cd apache-activemq-5.8.0/data/kahadb/
>> $ ls -lht | head
>> total 49G
>> -rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
>> -rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
>> -rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
>>
>> Based on the maximum log ID, almost none of the log files appear to
>> have been cleaned up:
>>
>> $ ls db-*.log | wc
>> 1540 1540 17421
>>
>> I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla
>> configuration is Java heap size, which shouldn't affect whether or not
>> DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with
>> Camel. Obviously from the number of queues, I have several different
>> workflows in flight, but here's a representative example of how I'm
>> using ActiveMQ in my Java code:
>>
>> from("activemq:statuslinks")
>>
>> .bean(new LinkExtract())
>>
>> .split(body(String.class).tokenize("\n"))
>>
>> .filter(new LinkHandled(jdbcConnectionString))
>>
>> .to("activemq:linkunwind");
>>
>>
>> The only problems I'm seeing in my activemq.log file appear to be
>> "normal." I get rashes of transport exceptions every few weeks, which
>> I have associated (rightly or wrongly) with hitting the web admin
>> interface. Given the infrequency and nature of the errors, I don't
>> think they explain why the logs wouldn't be deleted. In case it's
>> useful, though, here are a couple of representative lines from the log
>> file:
>>
>> 2013-12-13 09:46:52,269 | WARN | Transport Connection to:
>> tcp://127.0.0.1:39604 failed:
>> org.apache.activemq.transport.InactivityIOException: Channel was
>> inactive for too (>30000) long: tcp://127.0.0.1:39604 |
>> org.apache.activemq.broker.TransportConnection.Transport |
>> ActiveMQ InactivityMonitor Worker
>> 2013-12-13 14:43:41,215 | WARN | Transport Connection to:
>> tcp://127.0.0.1:35711 failed: java.io.EOFException |
>> org.apache.activemq.broker.TransportConnection.Transport |
>> ActiveMQ Transport: tcp:///127.0.0.1:35711@61616
>>
>>
>> Given the use case ‹ very simple message processing using Camel ‹ and
>> ActiveMQ's vanilla configuration, I would expect the log files to be
>> tiny. Clearly, though they're not. Does anyone have any idea why those
>> files aren't getting deleted? I'm sure I'm doing (or not doing)
>> something very silly, but after pouring over the logs and
>> documentation, I just can't figure out what I'm doing wrong. Does
>> anyone have an idea why my logs wouldn't be getting cleaned up?
>>
>> Thanks everyone in advance for your attention and help! And thank you
>> ActiveMQ devs for such a wonderful product! :)
>>
>> *Andy Boothe*
>>
>> ****
>Have you referenced this documentation:
>http://cp.mcafee.com/d/FZsS93gs839J5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5
>AsYeKCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7
>fc9L9CzBXBHFShhlLtzBgY-F6lK1FJ4SMrLOtXTLuZXTdTdw0ZJehDRhrJmSNf-00Uo9ByW5iV
>niuE42VZi9tSh_QbNZnpgbWXfUBX7_I6RK1NJKDNEnzYSUMbIzCnXBZ8lHcDVNyN4Nm9W7bwHr
>jApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPDa14Qgmf_ouwq86Rzh
>0nbCy01ipoArKruq0ctEi81xx
>
>
>--
>Tim Bish
>Sr Software Engineer | RedHat Inc.
>tim.b...@redhat.com |
>http://cp.mcafee.com/d/k-Kr6xEp3xESyOYUqenSbFECXCQQmhPMWWrMUSCyOeuuK-PsSCy
>Oeu7njuudFEIzDDArzC3uAaRCjY56b4j5oDEsKrcm8CaNfgVsSMCwOgRd_HYCOYYOVRXBQrICz
>DC4TAPhOZORQX8EGTKNOEuvkzaT0QSCrodTVeZXTLuZXCXCODeJQ-BmUS_BPrtfzgL7VJNwnp7
>cLTbWgHmpfPz5y9yIjQen1mSD8PWEJSHroD_00joVZZ4S2_id405qjQe6jh0vkyq87p7_gQggH
>cQg4vDek29EwIv-MZ0QgdH6y0Knd402AON8TsS5aMcvwY |
>http://cp.mcafee.com/d/2DRPoAd6QmnD3hO-Ntd4TsSCyOeu7nju76QQmhPPRTSrCQQmhPM
>WWrPNJd5AsYYzssMrQxmIOvwENoyoH4Z3BPpyN4Nm9W7bCS4Q6i6FLZvASnDCneLsKztAQsYMC
>YCqenKmKDp55mZSel3PWApmU6CSjr1K_9TLuZXTLsTsSkWfE4ZDOVJKDNEnzYSUMbIzCnXBZ8l
>HcDVNyN4Nm9W7bwHrjApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPD
>a14Qgmf_ouwq86Rzh0nbCy01ipoArKrZzCB4t6l
>skype: tabish121 | twitter: @tabish121
>blog:
>http://cp.mcafee.com/d/FZsS73hJ5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5AsYe
>KCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7fc9L
>9CzBXBHFShhlLtzBgY-F6lK1FJwSMrLOtXTLuZXTdTdw0VOl8iEjUwvkzhqI-nMSTjUQbN-rso
>5ShPbZO-AaRCjYUNoyoH4Z3BMlJFOc-GbtGSS9_M04SevvhdwLQzh01mAZ3xAQg7R8Cy1Sh_Qd
>44aPd417VPB0yq8b7_Ifgd43qNEwbBPh00FcIidTdKLPH3zP8
>
