KahaDB's data files are not compacted, so a tiny message can keep alive a 32-MB (or whatever size you're using) log file. It's not that the messages are larger, it's just that lots of clutter gets kept along with the actual content you're keeping. And if you're not using 5.14.0, then chains of files can be kept alive where only the first file contains live messages and the later files contain just acks for messages in the older files (which can't be discarded, or else the messages they ack will come back to life).
Either way, you should either implement a strategy for addressing DLQed messages in a timely manner or if that's not possible then configure your broker to set a JMSExpirationTime on each message or use the discarding dead letter strategy (as described in http://activemq.apache.org/message-redelivery-and-dlq-handling.html) or something similar to automatically dispose of DLQ messages if you're sure you're not going to use them anyway. Tim On Tue, Sep 27, 2016 at 8:27 AM, Johan Carlquist <jo...@su.se> wrote: > Thanks for your reply. > > > On 27 Sep 2016, at 14:59, Tim Bain <tb...@alumni.duke.edu> wrote: > > > > Do the stats you see in JConsole line up with KahaDBJournalReader (what's > > that?), or with the web console? And do you see an open XA transaction > in > > JConsole, like Steve did? > > No traces of XA transactions. > > > I'd guess that the negative values indicate that whatever code you > > hacked'n'slashed has a bug, either in the code you started with or in the > > code you changed/added. But that's pure guesswork. > > Highly possible. > The java isn't strong in this one… > > > The interesting part of your email (for me) is the sentence "the only > thing > > that seems correct is the amount of messages in > > ActiveMQ.DLQ.Queue.queue1.test," > > which implies that you know you have messages in the DLQ and aren't doing > > anything about it. By any chance, are one or more of those messages > > exactly as old as your oldest data file? If you purge that queue (and > all > > other DLQs), do your data files disappear? > > Yes. The DLQs haven't been reciving a lot of love since this is a > staging environment. > I thought I tested to purge the DLQs, but apperantly not. Did it now and > all the log files disappeard and freed a lot of disk. Maybe I didn't > wait until the gc before checking the disk usage last time. > > This might be a dumb question, but it feels like the messages in DLQ > took a lot more storage then "regular" messages. Is this since they are > spread over many log files? Is it possible to "defrag" the log files > without purging the queues? > > > > enableAckCompaction only compacts acks, not unconsumed messages. It will > > eliminate any data file being kept solely for acks, but by definition > > that's not your first data file (that one must have an unconsumed > message) > > and if you have unconsumed messages in later files it won't compact them > > either. It might be a problem if your goal is to have fewer journal > files, > > but you're OK with still having some, but if you want to get rid of all > the > > old ones, you need to make sure you're not holding old unconsumed > messages. > > Ok! My goal is to not fill disk. So it seems like handling the DLQ is a > better way to do it. > > -- > jocar > >
