Hi Torsten,
yea i tried with optimizedDispatch="false" also.
And i am using queue.
Here is my kahaDb settings :
<persistenceAdapter>
<kahaDB directory="data/kahadb" concurrentStoreAndDispatchQueues="false"
enableIndexWriteAsync="true" enableJournalDiskSyncs="false"
maxAsyncJobs="50000" cleanupInterval="60000" checkpointInterval="120000"
journalMaxFileLength="1g" indexCacheSize="300000"
indexWriteBatchSize="2147483647" />
</persistenceAdapter>
And i also tried with enableIndexWriteAsync="false" but nothing changes much.
Also when i ran profiler then i found out when the consumer starts for the
first time the producer thread goes in to waiting state for the lock on the
AbstractQueuedSynchronizer ( the queue it is supposed to write in).
Actually the problem is the degradation in message production rate . I can work
with low production rate but what is happening is due to consumers regularly
going up and down the production rate actually goes to near Zero value.
Right now i am using Vmconnector for both producer and consumer.Do you have any
suggestion here.What is the best choice .( I am thinking of using NIO for
producers).
T&R
Harish Sharma
On Dec 21, 2011, at 2:43 PM, Torsten Mielke wrote:
> Hard to say what exactly the problem is. In the past I got to learn the
> optimizedDispatch="true" isn't always a good choice. Can you try without it?
>
> In addition I recently found that enableIndexWriteAsync="true" on your kahadb
> config may not bring any performance improvement. Not sure if you have set
> this.
>
> Also does your test involve queue or topic messages?
>
>
> Torsten Mielke
> tors...@fusesource.com
> tmie...@blogspot.com
>
>
>
> On Dec 20, 2011, at 10:37 AM, Harish Sharma wrote:
>
>> Hi,
>> I tried to reply to
>> http://activemq.2283324.n4.nabble.com/Backlog-data-causes-producers-to-slow-down-tt3806018.html
>> post . Because i am facing similar issue.
>> Whenever my consumer goes down and there is a data back log the production
>> rate goes down significantly.
>> Moreover when my consumer is up again , the production rate goes down again
>> and this cycle repeats itself.
>> Some Info :
>> I am using ActiveMQ 5.5.1 , KahaDB message store, Producer flow control =
>> FALSE , AsyncSend(true) , persistence = true and
>> ConcurrentQueueStoreAndDispatch = true and false (tried both does not make
>> much difference)
>> As i read in this post about cursors so i am using the default store cursor.
>>
>> Here is snippet from broker.xml
>> destinationPolicy>
>> <policyMap>
>> <policyEntries>
>> <policyEntry topic=">" optimizedDispatch="true" memoryLimit="1gb"
>> producerFlowControl="false" />
>> <policyEntry queue=">" optimizedDispatch="true" memoryLimit="1gb"
>> producerFlowControl="false" />
>> </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>> <systemUsage>
>> <systemUsage sendFailIfNoSpace="true">
>> <memoryUsage>
>> <memoryUsage limit="20 gb" />
>> </memoryUsage>
>> <storeUsage>
>> <storeUsage limit="500 gb" name="foo" />
>> </storeUsage>
>> <tempUsage>
>> <tempUsage limit="1 gb" />
>> </tempUsage>
>> </systemUsage>
>> </systemUsage>
>>
>> I am using all the optimizations(i read from
>> http://fusesource.com/docs/broker/5.4/tuning/index.html )
>> for producer , consumer and broker.
>> These optimizations are able to increase message production rate a bit but
>> the main issue is the stability of the production rate.(why it goes down 10
>> times and then keep on going down with every time consumers goes down or
>> goes up again).
>> I am trying to figure out the cause but unable to pinpoint the code in
>> activeMQ which is causing this issue.
>> When i saw threads state in jconsole i found out that as soon as consumer
>> start running , producer thread goes into wait state ( State: WAITING on
>> java.util.concurrent.locks.AbstractQueuedSynchronizer ).I understand the
>> queue these producers write to is blocking queue.But i don't understand why
>> consumers are affecting producers while flow control is off and my store
>> size is 500 GB and memory is 20GB .Why not producers are producing at a
>> constant rate until my KahaDB store is full.
>>
>> I really need help in this matter and deeply appreciate any help from you
>> guys.
>>
>> Thanks & Regards
>> Harish Sharma
>
>
>
>
T&R
Harish Sharma
