Auto batching does not exist at the moment but it would be a nice enhancement. ie: configure some sort of client side pause time or size limit so that the transport would wait for a batch and send all at once in one compressed block. It would require an update to openwire to indicate the start/end of a compressed batch or a new batch command, but it is doable. Please open a jira ticket to capture this. It definatly warrents some investigation. Either it is something that activemq can do under the hood or something we can implement via plugins and filters.
On 14 November 2012 16:41, Ken1000 <kendrum...@yahoo.com> wrote: > We've been using ActiveMq (as well as ServiceMix and Camel) for some time > now, and they all work great handling XML data (we generally use > non-persistent, non-transacted topics). One area that does cause us > problems is the bandwidth when transmitting the data over a wire. Sometimes > we're sending a lot of relatively small XML messages, with a handful of JMS > string properties describing the contents of each message. > > We specify in our Camel configuration that the ActiveMQ broker should > compress the messages before sending, but they don't seem to compress down > very well because 1) we think the JMS properties aren't compressed, and 2) > they are relatively small messages without a lot of repetition in the text. > > We've tried using the Camel "aggregator" pattern to batch message bodies > together before sending, which gives much better compression, but at the > cost of losing the properties attached to each message. We could avoid > losing the properties by 1) aggregating messages based on common properties > (which results in smaller batches and less compression - an undesired > result), or 2) by aggregating each message's properties along with the > message bodies (which could then be used to reconstitute the individual JMS > messages with their properties after transmission), but it seems this would > require any consumer to have a complimentary de-aggregator in place - > another undesired result. > > Our dream solution would be if the ActiveMQ broker could be configured to > batch and compress messages (including the properties) prior to > transmission, and then separate them back to individual messages to be read > by the subscriber's onMessage call, but I can't find this anywhere in the > documentation (prefetch doesn't seem to handle this, etc...). > > Is there something I've missed that would enable this? > > Thanks! > > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Batching-messages-for-better-compression-tp4659340.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. -- http://redhat.com http://blog.garytully.com
