Thank you for your comments Joe, both senerios you described result in a consumer/session being flooded with messages up to the prefetch limit. I guess the transacted multi-threaded client code would need to evenly iterate the sessions looking for whatever sessions have the messages. Setting the prefetch limit low shoud ensure a large number of messages are distributed across the sessions while a low number of messages can reasonabily route to a single session where all messages get processed in a single thread.
I am pooling my consumer/sessions and using each one to receive a message and use it within a thread. Problem is, Apache Commons Pooling does not allow iteration of items in a pool - to find the session with all the messaages. What is the lowest prefetch limit that would ensure good throughput of messages without pulling too often from the broker terry k ttmdev wrote: > > Here's what I have experienced wrt prefetch limits; you may find it > useful. > > If a destination (queue) has active consumers, then as messages arrive at > the destination, the broker will evenly dispatch the messages across the > active consumers; no matter what the prefetch limit may be. > > Now let's say that you have a destination that has a bunch of pending > messages, but no active consumers to send them to. If you then activate > say 5 consumers for that destination, the broker will latch on to the > first of those 5 that connects to it and will stream the pending messages > - up to the prefetch limit - to that first consumer. As soon as it > dis-engages from the first one, it will then begin to 'evenly' dispatch > messages to all 5 (i.e., assuming the other 4 have already connected w/the > broker). If the prefetch limit is set to a value that is higher than or > equal to the number of pending messages, then that first consumer will get > ALL the messages. > > Since your message processing time is relatively small, you shouldn't have > to have a prefetch limit of '1'. > > Hope this helps, > Joe > > > > > Terry K wrote: >> >> In a point-to-point queue I want to use transacted sessions to ensure >> messages are processed completely before being removed from the queue. >> The client is to be multi threaded to process messages in parallel. >> >> I decided each thread would need it's own Session object (for >> transacting) so that when the work on each message is successfully >> completed the session can be commited. Session objects would be pooled >> and retrieved when the next message is ready to be received. >> >> The documentation states that, "ActiveMQ is a high performance message >> bus that streams messages to consumers as fast as possible so that the >> consumer always has a buffer of messages, in RAM, ready to process - >> rather than have them explicitly pull messages from the server which adds >> significant latency per message". >> >> The problem is that messages will not be evenly distributed amoungst >> Sessions, with the likeihood that a single session will be flooded with >> all the messages making it impossible to distribute the work amoungst >> multiple threads. >> >> The logical next step would be to set the prefetch limit to one and have >> a pull-one-message-at-a-time dispatch policy (as described >> here:http://activemq.apache.org/what-is-the-prefetch-limit-for.html). >> Apparently, this is advised against as it's very SLOW >> (http://osdir.com/ml/java.activemq.user/2005-03/msg00188.html). I expect >> millions of messages in my queue, each one can be processed in a matter >> of seconds. >> >> The problem of multi threading and transacted sessions must be fairly >> common - can anyone suggest other options? >> >> Terry >> > > -- View this message in context: http://www.nabble.com/transacted-sessions-in-multi-threaded-client-tf4603908s2354.html#a13166649 Sent from the ActiveMQ - User mailing list archive at Nabble.com.
