Re: OOM with high KahaDB index time

2010-02-19 Thread rideallday
That was it. Thanks! I had to set to indexCacheSize = 10 to get the performance I'm looking for. rajdavies wrote: > > ok - it looks like you might need to tune KahaDB - set the > indexCacheSize = 1000 - see http://activemq.apache.org/kahadb.html > > > -- View this message in conte

Re: OOM with high KahaDB index time

2010-02-17 Thread Rob Davies
ok - it looks like you might need to tune KahaDB - set the indexCacheSize = 1000 - see http://activemq.apache.org/kahadb.html cheers, Rob On 15 Feb 2010, at 18:46, rideallday wrote: Yes, topics are durable. Test case: 1. publish messages to queue, let several hundred thousand accumulate. 2

Re: OOM with high KahaDB index time

2010-02-15 Thread rideallday
Yes, topics are durable. Test case: 1. publish messages to queue, let several hundred thousand accumulate. 2. publish topics to queue, let several hundred thousand accumulate 3. begin consuming from queue. 4. begin consuming from topic. You should see a huge degrade in performance and a bunch of

Re: OOM with high KahaDB index time

2010-02-12 Thread Rob Davies
could you provide a test case ? cheers, Rob On 11 Feb 2010, at 21:41, rideallday wrote: Umm, actually I'm still seeing issues with high Index times for topics. My test case is: Send a huge amount of messages into the broker. Let several hundred thousand accumulate. Then start consumers

Re: OOM with high KahaDB index time

2010-02-11 Thread Rob Davies
Are you using Durable Topic Consumers ? On 11 Feb 2010, at 21:41, rideallday wrote: Umm, actually I'm still seeing issues with high Index times for topics. My test case is: Send a huge amount of messages into the broker. Let several hundred thousand accumulate. Then start consumers and

Re: OOM with high KahaDB index time

2010-02-11 Thread rideallday
Umm, actually I'm still seeing issues with high Index times for topics. My test case is: Send a huge amount of messages into the broker. Let several hundred thousand accumulate. Then start consumers and see how long it takes to drain the queues and topics. The result is: For queues- excellent.

Re: OOM with high KahaDB index time

2010-02-01 Thread rideallday
I can confirm that has resolved the issue I was seeing. Thanks Rob! What's ETA for 5.4 or applying this fix in 5.3? I'll need an official stable release to put into production. -- View this message in context: http://old.nabble.com/OOM-with-high-KahaDB-index-time-tp27217704p27413891.html Sent

Re: OOM with high KahaDB index time

2010-02-01 Thread Rob Davies
better. Thanks for the help! -Original Message- From: Gary Tully [mailto:gary.tu...@gmail.com] Sent: Wednesday, January 27, 2010 2:35 AM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time Hi Daniel, builds from today are there now: https://repository.apache.org

RE: OOM with high KahaDB index time

2010-02-01 Thread Daniel Kluesing
Ran this through some paces, and it works much better. Thanks for the help! -Original Message- From: Gary Tully [mailto:gary.tu...@gmail.com] Sent: Wednesday, January 27, 2010 2:35 AM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time Hi Daniel, builds from today

Re: OOM with high KahaDB index time

2010-01-27 Thread Gary Tully
AM > To: users@activemq.apache.org > Subject: Re: OOM with high KahaDB index time > > Thanks Daniel! Though please wait for the next valid 5.4 build - looks > like its missing a dependency at the moment > On 21 Jan 2010, at 16:48, Daniel Kluesing wrote: > > > I

Re: OOM with high KahaDB index time

2010-01-26 Thread Rob Davies
/ still lists the build date as 1/20) Once it's out, I'll give it a run through. -Original Message- From: Rob Davies [mailto:rajdav...@gmail.com] Sent: Thursday, January 21, 2010 9:42 AM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time Thanks Dani

RE: OOM with high KahaDB index time

2010-01-26 Thread Daniel Kluesing
@gmail.com] Sent: Thursday, January 21, 2010 9:42 AM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time Thanks Daniel! Though please wait for the next valid 5.4 build - looks like its missing a dependency at the moment On 21 Jan 2010, at 16:48, Daniel Kluesing wrote: >

Re: OOM with high KahaDB index time

2010-01-25 Thread Rob Davies
utions.com] Sent: Thursday, January 21, 2010 1:01 PM To: users@activemq.apache.org Subject: RE: OOM with high KahaDB index time Just for grins, I threw your cfg file into our 5.3 testbed and sure enough, we got the OOMs; I pumped 200k messages with each being 2k in size. FWIW, takin

Re: OOM with high KahaDB index time

2010-01-25 Thread Fred Moore
gt;> > So I'm not sure if that's what Rob is talking about being fixed in 5.4 > >> > (And I'll try the snapshot as soon as it's ready) but if I don't have > >> the > >> > sendFailIfNoSpace then my understanding is the producers send calls &g

Re: OOM with high KahaDB index time

2010-01-25 Thread Joe Fernandez
calls >> > block/wait/timeout - as opposed to fail - so it's more difficult to get >> > into a HA configuration. It's a minor point, not having an OOM is much >> > more important, but I definitely want the send calls to fail for the >> > producer if th

Re: OOM with high KahaDB index time

2010-01-25 Thread Fred Moore
t's a minor point, not having an OOM is much > > more important, but I definitely want the send calls to fail for the > > producer if the broker ever does anything funny. > > > > Thanks for the feedback on the config, very helpful. > > > > -Original Messag

Re: OOM with high KahaDB index time

2010-01-25 Thread Joe Fernandez
ducers send calls >>> block/wait/timeout - as opposed to fail - so it's more difficult to >>> get >>> into a HA configuration. It's a minor point, not having an OOM is >>> much >>> more important, but I definitely want the send calls to fail for the

Re: OOM with high KahaDB index time

2010-01-22 Thread Rob Davies
n the config, very helpful. -Original Message- From: Joe Fernandez [mailto:joe.fernan...@ttmsolutions.com] Sent: Thursday, January 21, 2010 1:01 PM To: users@activemq.apache.org Subject: RE: OOM with high KahaDB index time Just for grins, I threw your cfg file into our 5.3 testbed and sure en

Re: OOM with high KahaDB index time

2010-01-22 Thread Joe Fernandez
, not having an OOM is >>> much >>> more important, but I definitely want the send calls to fail for the >>> producer if the broker ever does anything funny. >>> >>> Thanks for the feedback on the config, very helpful. >>> >>>

Re: OOM with high KahaDB index time

2010-01-22 Thread Rob Davies
1, 2010 1:01 PM To: users@activemq.apache.org Subject: RE: OOM with high KahaDB index time Just for grins, I threw your cfg file into our 5.3 testbed and sure enough, we got the OOMs; I pumped 200k messages with each being 2k in size. FWIW, taking this out of the cfg file made things run a lot b

RE: OOM with high KahaDB index time

2010-01-22 Thread Joe Fernandez
to fail for the > producer if the broker ever does anything funny. > > Thanks for the feedback on the config, very helpful. > > -Original Message- > From: Joe Fernandez [mailto:joe.fernan...@ttmsolutions.com] > Sent: Thursday, January 21, 2010 1:01 PM > To: users@activemq

RE: OOM with high KahaDB index time

2010-01-21 Thread Daniel Kluesing
al Message- From: Joe Fernandez [mailto:joe.fernan...@ttmsolutions.com] Sent: Thursday, January 21, 2010 1:01 PM To: users@activemq.apache.org Subject: RE: OOM with high KahaDB index time Just for grins, I threw your cfg file into our 5.3 testbed and sure enough, we got the OOMs; I pumped 200k messages with

RE: OOM with high KahaDB index time

2010-01-21 Thread Joe Fernandez
> >producerFlowControl="true" > memoryLimit="10mb"> > > > > > > >

Re: OOM with high KahaDB index time

2010-01-21 Thread Rob Davies
memoryLimit="10mb"> -Original Message- From: Rob Davies [mailto:rajdav...@gmail.com] S

RE: OOM with high KahaDB index time

2010-01-21 Thread Joe Fernandez
> > > > > >producerFlowControl="true" > memoryLimit="10mb"> > > > > > &

RE: OOM with high KahaDB index time

2010-01-21 Thread Daniel Kluesing
t;; brokerName="sub01chi" dataDirectory="${activemq.base}/data"> -Original Message- From: Rob Davies [mailto:rajdav...@gmail.com] Sent: Monday, January 18, 2010 10:42 PM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB in

Re: OOM with high KahaDB index time

2010-01-21 Thread Rob Davies
esday, January 20, 2010 2:08 PM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time I have a potential fix for this problem - hoping someone will try out the next 5.4-snapshot :) On 20 Jan 2010, at 21:41, rideallday wrote: I went back to AMQStore. Working fine for me. --

RE: OOM with high KahaDB index time

2010-01-21 Thread Daniel Kluesing
I'm happy to give 5.4-snapshot a try -Original Message- From: Rob Davies [mailto:rajdav...@gmail.com] Sent: Wednesday, January 20, 2010 2:08 PM To: users@activemq.apache.org Subject: Re: OOM with high KahaDB index time I have a potential fix for this problem - hoping someone wil

Re: OOM with high KahaDB index time

2010-01-20 Thread Rob Davies
I have a potential fix for this problem - hoping someone will try out the next 5.4-snapshot :) On 20 Jan 2010, at 21:41, rideallday wrote: I went back to AMQStore. Working fine for me. -- View this message in context: http://old.nabble.com/OOM-with-high-KahaDB-index-time-tp27217704p27249114

Re: OOM with high KahaDB index time

2010-01-20 Thread rideallday
I went back to AMQStore. Working fine for me. -- View this message in context: http://old.nabble.com/OOM-with-high-KahaDB-index-time-tp27217704p27249114.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: OOM with high KahaDB index time

2010-01-20 Thread rideallday
rajdavies wrote: > > FileCursor and Kaha (which is > used by the AMQStore). - can you share the configuration you are using? > KahaDB isn't the same as Kaha > I'm still unsure... I'm using KahaDB, not AMQStore. I was about to go back to AMQStore as this setup just isn't working, but I reall

Re: OOM with high KahaDB index time

2010-01-19 Thread Rob Davies
Yes - those settings apply to the both FileCursor and Kaha (which is used by the AMQStore). - can you share the configuration you are using? KahaDB isn't the same as Kaha - we weren't very imaginative with names Im afraid On 19 Jan 2010, at 23:48, rideallday wrote: Rob, I'm seeing the exa

Re: OOM with high KahaDB index time

2010-01-19 Thread rideallday
Rob, I'm seeing the exact same behaviour as Daniel, > > So I would recommend the following > > 2. Try KahaDB - which - with the BTreeIndex - will not hit the > problems you are seeing with the Filecursor > > however I am already using KahaDB as the persistenceAdapter. Your advice seem

Re: OOM with high KahaDB index time

2010-01-18 Thread Rob Davies
On 18 Jan 2010, at 22:14, Daniel Kluesing wrote: Hi, I'm running the 5.3 release as a standalone broker. In one case, a producer is running without a consumer, producing small, persistent messages, with the FileCursor pendingQueuePolicy (per https://issues.apache.org/activemq/browse/AMQ-25