Hello,
I'd like to commit a large number of messages in a single transaction. A
typical number
of message is one million and each message size is 512 bytes.
So I have tested with commiting 100K messages at once. It worked with
changing the memoryLimit
to 200 MB in activemq.xml.
<policyEntry queue="queue.trigger" producerFlowControl="true"
memoryLimit="200mb">
In above configuration, the problem is uncommitted messages consume RAM.
Suppose when 1M messages are
committed at once, 2GB RAM is consumed by uncommited messages and this is
not acceptable.
Is it possible to configure the activemq storing uncommited messages to disk
instead of RAM ?
I am using the kahadb as a message store.
The followings are my test code.
- Sender.java
--------------------------------------------------------------------------------------------
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.TextMessage;
import javax.jms.Queue;
import javax.jms.Connection;
import javax.jms.MessageProducer;
import javax.jms.Session;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import javax.jms.Connection;
public class Sender {
Connection connection = null;
private static int STRING_SIZE = 512;
private static int LOOP_COUNT = 1000000;
public static void main(String[] args) throws Exception {
try {
if (args.length < 6) {
System.err.println("Please enter correct parameters");
System.err.println("parameters are broker-url username
password queue-name key1=value1 key2=value2 ...");
}
String url = args[0];
String user = args[1];
String password = args[2];
String queueName = args[3];
STRING_SIZE = Integer.valueOf(args[4]);
LOOP_COUNT = Integer.valueOf(args[5]);
ActiveMQConnectionFactory factory = new
ActiveMQConnectionFactory(user,password,url);
Connection connection = factory.createQueueConnection();
connection.start();
Session session =
connection.createSession(true,Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue(queueName);
MessageProducer producer = session.createProducer(queue);
producer.setDeliveryMode(javax.jms.DeliveryMode.PERSISTENT);
String text = "";
for (int j=0;j<STRING_SIZE;j++) {
text += "0";
}
for (int i=0;i<LOOP_COUNT;i++) {
System.out.println("count = "+ i);
TextMessage msg = session.createTextMessage();
msg.setText(text);
producer.send(msg);
}
session.commit();
producer.close();
session.close();
connection.close();
} catch (JMSException e) {
e.printStackTrace();
System.err.println("Something is wrong with JMS access.");
System.exit(1);
}
System.exit(0);
}
}
--------------------------------------------------------------------------------------------
- Sender.sh
--------------------------------------------------------------------------------------------
java -cp /usr/local/oss/apache-activemq-5.8.0/activemq-all-5.8.0.jar:.
Sender tcp://localhost:61616 system manager queue.trigger 512 100000
--------------------------------------------------------------------------------------------
--
View this message in context:
http://activemq.2283324.n4.nabble.com/commiting-a-large-number-of-messages-in-a-single-transaction-tp4668672.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
