Thanks for clarifying. On Feb 2, 2017 6:33 AM, "Christopher Shannon" < christopher.l.shan...@gmail.com> wrote:
> The broker will continue to work just fine and the index deletion is not > required. You only need to delete the index file if you want to reclaim > the wasted space otherwise the fix will at least prevent the issue from > getting worse. > > On Thu, Feb 2, 2017 at 8:27 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > > > Chris, > > > > What will be the behavior for anyone who doesn't delete their index after > > upgrading? Will the broker continue to function as-is in that scenario, > or > > is this a required procedure during an upgrade beyond 5.14.3? > > > > Tim > > > > On Feb 2, 2017 5:02 AM, "Christopher Shannon" < > > christopher.l.shan...@gmail.com> wrote: > > > > > Sorry for taking a while to get back to you. > > > > > > Based on your originally message and after analyzing some index files I > > > have actually discovered a bug with KahaDB that causes the index file > to > > > grow too large when things like out of memory errors occur or any > unclean > > > shutdown. On recovery the free pages in the index file are not being > > > properly tracked which is leading to excessive index file usage. So > this > > > is probably the issue that you are seeing. I will be fixing this in > > > version 5.14.4. Upon upgrade you will need to delete the old index and > > > have it be rebuilt and then going forward it should work as intended. > > You > > > can track the status of the bug here: > > > https://issues.apache.org/jira/browse/AMQ-6590 > > > > > > In terms of the in memory cache, only a fixed number of pages are > cached. > > > The default is 10 thousand pages and each page is 4KB. So roughly 40 > MB > > of > > > memory will be used at most. If the number of pages in use grows > beyond > > > that then the cache will swap pages based on the defined eviction > > > strategy. You can increase the number of pages in memory (which would > > help > > > performance at the cost of eating up memory) by tweaking the > > indexCacheSize > > > value of KahaDB. See http://activemq.apache.org/kahadb.html for more > > > info. > > > > > > On Tue, Jan 24, 2017 at 11:00 PM, Scorpio <n...@xiangjiabao.com> wrote: > > > > > > > Christopher: > > > > > > > > > > > > Thank you for your explanation! > > > > > > > > Hope there would be other proper ways to shrink the index file in > > future > > > > ActiveMQ releases. > > > > > > > > Once we monitored that our broker is killed by system due to "out of > > > > memory". > > > > > > > > I know that meta cache in memory will be synced into index file at > > > > checking points. _*Does this mean there is as much memory as the > > > > db.data file being used?*_ > > > > > > > > Even though our broker is idle, we still find that it uses up much > > > > memory. Could they be relevant associated? Thanks > > > > > > > > > > > > Kind regards, > > > > > > > > Scorpio > > > > > > > > On 2017/1/24 20:16, christopher.l.shannon [via ActiveMQ] wrote: > > > > > The index grows based off the amount of data it tracks in the > > journal, > > > > > but doesn't shrink. This means if you had a lot of data in your > > > > > journal at some point (several gigabytes, etc) then the index would > > > > > have grown in size with enough page files to keep track of all of > the > > > > > data. This is usually not an issue because the index size > generally > > > > > stabilizes at some point relative to the work load of the broker. > > If > > > > > you really want to shrink it the only way right now is to stop the > > > > > broker, delete the index and restart so it does a replay and > rebuilds > > > > > the index but the size would just grow again as messages flow. > > > > > > > > > > On Mon, Jan 23, 2017 at 8:06 PM, Scorpio <[hidden email] > > > > > </user/SendEmail.jtp?type=node&node=4721300&i=0>> wrote: > > > > > > > > > > > Hi, All, > > > > > > > > > > > > In my activemq instance, I find that the kahadb index file is > > larger > > > > > than > > > > > > the journal file. > > > > > > > > > > > > As far as I know, /db.data/ is the meta store, index for the > > journal > > > > > file, > > > > > > which should means it is much smaller than the journal file. > > > > > However, it > > > > > > does not look like that. The index file seems to keep growing > while > > > the > > > > > > journal file is rotated. Do I miss something? Thanks. > > > > > > > > > > > > Log attached: > > > > > > root@host:~/run/apache-activemq-5.13.1/data/kahadb# du * -sh > > > > > > 16M db-16.log > > > > > > 1.2G db.data > > > > > > 3.2M db.redo > > > > > > 4.0K lock > > > > > > root@host:~/run/apache-activemq-5.13.1/data/kahadb# > > > > > > > > > > > > Kind regards, > > > > > > Scorpio > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > View this message in context: > > > > > http://activemq.2283324.n4.nabble.com/kahadb-index-file- > > > > larger-than-journal-file-tp4721284.html > > > > > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ------------ > > > > > If you reply to this email, your message will be added to the > > > > > discussion below: > > > > > http://activemq.2283324.n4.nabble.com/kahadb-index-file- > > > > larger-than-journal-file-tp4721284p4721300.html > > > > > > > > > > To start a new topic under ActiveMQ - User, email > > > > > ml-node+s2283324n2341805...@n4.nabble.com > > > > > To unsubscribe from kahadb index file larger than journal file, > click > > > > > here > > > > > <http://activemq.2283324.n4.nabble.com/template/ > > NamlServlet.jtp?macro= > > > > unsubscribe_by_code&node=4721284&code=bnlAeGlhbmdqaWFiYW8uY29tfDQ3Mj > > > > EyODR8MTkzNzMzOTQ5NQ==>. > > > > > NAML > > > > > <http://activemq.2283324.n4.nabble.com/template/ > > > > NamlServlet.jtp?macro=macro_viewer&id=instant_html% > > > > 21nabble%3Aemail.naml&base=nabble.naml.namespaces. > > > > BasicNamespace-nabble.view.web.template.NabbleNamespace- > > > > nabble.view.web.template.NodeNamespace&breadcrumbs= > > > > notify_subscribers%21nabble%3Aemail.naml-instant_emails% > > > > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > View this message in context: http://activemq.2283324.n4. > > > > nabble.com/kahadb-index-file-larger-than-journal-file- > > > > tp4721284p4721315.html > > > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > > > > > > > > >
