Hello, We are evaluating the possible use of ActiveMQ in our projects and have encountered this problem.
When sending small messages (for example, ByteMessages with a 1KB byte array) from a producer to a consumer through a broker, there is much higher latency than with bigger messages (for example, ByteMessages with a 32KB byte array), using the TCP transport. Changing the socketBufferSize parameter, the tcpNoDelayEnabled flag does not help. Version of ActiveMQ is 4.1, and in 4.0 there is also this problem. Our test is as follows: one JVM is the broker, the other JVM runs a producer and a consumer in separate threads (with separate connections and sessions). We are using a queue, no persistence, no transactions, Session.AUTO_ACKNOWLEDGE. The producer sends a message just after taking note of the time with System.nanoTime(), and then waits on a semaphore. When the consumer receives the message (implementing the onMessage() method of the MessageListener interface), the first thing that is done is taking notice of the time with System.nanoTime(). The difference in time is computed and written to a file, after which the consumer releases the semaphore so that the producer sends the next message. With 1 machine running Red Hat EL4 Update 3, hosting both the broker process and the producer/consumer process, we get a latency of 39ms with 1KB ByteMessages and latency of 0-1ms with 32KB ByteMessages. The frontier for latency appearing seems to be around 16KB. With 1 machine running WinXP Pro SP2, hosting both the broker process and the producer/consumer process, we get no latency (0-1 ms) with messages of any size between 1KB and 32KB. With 2 machines in a LAN, no matter the combination of WinXP / Linux (even 2 WinXP machines) we get high latency (even 200 ms for 2 WinXP machines) with 1KB messages and much lower latency (not 0, but low) with 32 KB messages. Some more facts: The first 1KB message has high latency (as expected due to initialization of stuff), the next 8-20 have no latency, and then the rest have high latency. In Linux localhost case, if the producer waits 1 second between sending 1KB messages, latency is low. Have not tried in other scenarios. Normally we do tests restarting the broker each time. But, if we do not restart the broker, the results can be more strange. Sometimes, there is a pattern where a 1KB message has no latency, the next had 39ms latency, the next has no latency, the next has 39ms latency, etc. In one of these times we took an Ethereal trace and found that the producer was sending 2 messages in the same TCP packet of around 2KB size. The broker configuration file is: http://www.nabble.com/file/6205/activemq.xml activemq.xml The code to perform the test is: http://www.nabble.com/file/6207/LatencyTest.java LatencyTest.java Here are some logs we have taken: http://www.nabble.com/file/6206/logs.zip logs.zip For 1KB messages you also have ethereal logs (32KB message ethereal logs are too big files). In every case, the file "client.txt" is the log of the producer/consumer process, the file "broker.txt" is the log of the broker process. Both these logs use the log4j logger of ActiveMQ. The "nolog" folders have logs without use of the log4j logger, since printing lots of messages disturbs a bit the results. Then you have a file called for example 1BytesMessage.txt (stands for 1KB messages), with the latency times and time of receiving by the consumer. All this is very strange... we do not know if the problem lies in the Linux TCP stack, the Java I/O or networking libraries, ActiveMQ code, some lack of proper configuration... we believe however that it is the producer which is holding sometimes a message before sending the packet to the broker. -- View this message in context: http://www.nabble.com/High-latency-for-small-messages-problem-tf3159901.html#a8764113 Sent from the ActiveMQ - User mailing list archive at Nabble.com.
