One more question: will the same thing happen if I switch to leveldb? 2015-02-28 22:53 GMT+08:00 Rural Hunter <ruralhun...@gmail.com>:
> I'm sorry I made a mistake. My storage is kahadb. We switched from leveldb > to kahadb a while ago and I forgot that. > Thanks for the links. Now understand what happened! > > 2015-02-28 19:03 GMT+08:00 Tim Robbins <tim.robb...@outlook.com>: > >> Hi, >> >> Two suggestions for you: >> >> 1. Try decreasing the logSize parameter for LevelDB. You’ve have a >> greater number of smaller log files, and a greater chance of each log file >> being garbage-collected. >> 2. With KahaDB, it’s possible to configure multiple KahaDB stores, and to >> put your dead-letter type messages into a different store than everything >> else to reduce overhead: >> http://blog.garytully.com/2011/11/activemq-multiple-kahadb-instances.html >> < >> http://blog.garytully.com/2011/11/activemq-multiple-kahadb-instances.html >> > >> Unfortunately it doesn’t appear that this applies to LevelDB yet! >> >> Regards, >> >> Tim >> >> > On 28 Feb 2015, at 7:27 pm, Rural Hunter <ruralhun...@gmail.com> wrote: >> > >> > Hi, >> > >> > Activemq version 5.10.2, storage: leveldb. >> > >> > I have a queue which serves similiar function as dead letter queue. My >> > application process messages from another queue and if the processing >> > fails, it put the message into this queue. The messages are persistent >> and >> > average several KBs in size. My application processes many messages but >> the >> > failed message count is very small, less than 100 a day. I noticed after >> > the application running for several days, my activemq storage becomes >> > almost full. I configured the storage to 30G. I checked the normal >> queues >> > and topics and there is no queue with large count of message. Most of >> them >> > are empty and some have only several messages. Only the failure message >> > queue I metioned above has a few hundred messages(about 500) which are >> > accumulated in several days. >> > >> > I have no idea what takes so much storage. I checked the storage files >> and >> > found there are many db-xxxx.log with timestamp almost through the >> several >> > days. They are not consequent though. Some of the db-xxx.log files are >> not >> > there. So the file list is like this: >> > db-1000.log >> > db-1001.log >> > db-1003.log >> > db-1004.log >> > db-1005.log >> > db-1008.log >> > ... >> > I suspect the failed messages are in those db-xxx.log files so I just >> tried >> > to clear the failed message queue. Right after that I found those old >> > db-xxx.log disappeared and the storage usage went back to 2%. So it >> seems >> > clear that the about 500 failed messages took around 30G storage. But >> how >> > can it be? Those messages are very small in size and the total size of >> the >> > messsage should be no more than a few MBs. >> >> >
