Hi, I have a producer that must be able to keep producing even when the remote broker is unavailable. failover is not enough since the problem might be in the network or even the local NIC.
Thus, I planed to have a local broker that will use a network connector to route all messages to the remote broker. I use ActiveMQ 5.10 with levelDB persistence on Java SE 1.7 (both local and remote servers). Average message length is 1KB. I need to be able to send at least 10,000 message per second. Send throughput to the local broker is about 100 messages per second. When using transactions and committing every 1000 messages, throughput jump to 12,000 messages per second. The network connector doesn't use transactions. Thus, its throughput is only 100 messages per second. I tried to use proxy connector in order to force useAsyncSend=true on the network connector with no success. Finally, I removed the network connector all together and implemented its functionality myself. In another thread on the local server I used a transacted session to local broker plus a transacted session to the remote broker. Each message received from the local broker was sent into the remote broker. Every 1000 messages (or when more than a second passed since last commit), I committed both sessions. The throughput of the new forwarder thread is 11,000 message per second. When running in parallel to the local producer the throughput is still 11,000 messages per second. I would like to suggest two additional properties to networkConnector: 1. maxBatchSize - if specified and higher than 1, use transactions to forward messages to remote broker. 2. maxLatency. The network connector will send (commit) up to <maxBatchSize> messages in a transaction. It will commit immediately, if more than <maxLatency> milliseconds passed since last commit. -- View this message in context: http://activemq.2283324.n4.nabble.com/Network-Connector-Performance-tp4690186.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
