What are your systemLimits, especially your storeUsage and memoryUsage? Why is producerFlowControl="false"? The sole purpose of PFC is to prevent the broker from OOMing when given more messages than it can store; you should have this on.
What are the units for the number 40000 that you quoted? Bytes? Lines? Is the OOM happening after 500 lines of a single file (or of a few files), or after around 500 files (each of which produces many messages)? I think you said the former, but want to be sure. How large is each line (and therefore how large is each message)? What happens if you temporarily increase the heap to 1GB? How many messages does it take in that scenario? Tim On Mar 25, 2016 9:00 AM, "Michele" <michele.mazzi...@finconsgroup.com> wrote: > Hi everyone, > > I'm new in using ActiveMQ and according to business requirement I have a > Camel Route optimized to read large size file (~40000) to split and store > each single row in AMQueue and then a pool consumer will process invoking a > RS. > > ActiveMQ after about 500 messages go down in OutOfMemory - Heap Space. > > I use JBossFuse ESB 6.2 based on ActiveMQ 5.11 and according to > http://activemq.apache.org/javalangoutofmemory.html I modified > activemq.xml > in installationJBFuseDir/etc but the problem continue: > > <policyEntry queue=">" producerFlowControl="false" > optimizedDispatch="true" queuePrefetch="0"> > <pendingMessageLimitStrategy> > <constantPendingMessageLimitStrategy limit="1000"/> > </pendingMessageLimitStrategy> > <deadLetterStrategy> > > <individualDeadLetterStrategy queuePrefix="Test.DLQ."/> > </deadLetterStrategy> > <pendingQueuePolicy> > <storeCursor /> > </pendingQueuePolicy> > </policyEntry> > > <persistenceAdapter> > > <levelDB directory="${data}/leveldb" /> > </persistenceAdapter> > > And I added jvm properties > -Xmx512M -Dorg.apache.activemq.UseDedicatedTaskRunner=false > > Errors: > > PooledConnectionFactory - Expiring connection ActiveMQConnection > > {id=ID:FGBAL201530-50934-1458820732064-7:3,clientId=ID:FGBAL201530-50934-1458820732064-6:2,started=false} > on IOException: Unexpected error occurred: java.lang.OutOfMemoryError: Java > heap space > > or > > Ignoring no space left exception, java.io.IOException: Java heap space > java.io.IOException: Java heap space > at > > org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:39)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:552)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:1044)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1357)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1271)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:735)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:860)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:381)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1896)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2107)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.broker.region.Queue.iterate(Queue.java:1583)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)[184:org.apache.activemq.activemq-osgi:5.11.0.redhat-620133] > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_79] > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_79] > at java.lang.Thread.run(Thread.java:745)[:1.7.0_79] > Caused by: java.lang.OutOfMemoryError: Java heap space > at org.fusesource.hawtbuf.Buffer.<init>(Buffer.java:42) > at > org.apache.activemq.leveldb.RecordLog$LogReader.read(RecordLog.scala:380) > at > > org.apache.activemq.leveldb.RecordLog$$anonfun$read$2.apply(RecordLog.scala:654) > at > > org.apache.activemq.leveldb.RecordLog$$anonfun$read$2.apply(RecordLog.scala:654) > at > org.apache.activemq.leveldb.RecordLog.get_reader(RecordLog.scala:644) > at org.apache.activemq.leveldb.RecordLog.read(RecordLog.scala:654) > at > > org.apache.activemq.leveldb.LevelDBClient.getMessage(LevelDBClient.scala:1335) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1274) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1271) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1359) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1358) > at > > org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:323) > at > > org.apache.activemq.leveldb.LevelDBClient$RichDB.cursorRange(LevelDBClient.scala:325) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply$mcV$sp(LevelDBClient.scala:1358) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1358) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1358) > at > > org.apache.activemq.leveldb.LevelDBClient.usingIndex(LevelDBClient.scala:1038) > at > > org.apache.activemq.leveldb.LevelDBClient$$anonfun$might_fail_using_index$1.apply(LevelDBClient.scala:1044) > at > > org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:549) > at > > org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:1044) > at > > org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1357) > at > > org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1271) > at > org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:735) > at > > org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:860) > at > > org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109) > at > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:381) > at > > org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142) > at > > org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159) > at > > org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1896) > at > org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2107) > at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1583) > at > > org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) > > How to configure Broker to handle the load? > > Thanks in advance > > Best Regards > > Michele > > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Active-MQ-OutOfMemory-in-JbossFuse-6-2-context-tp4709960.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
