Let me give a clearer example. Lets say I have 4 "real" messages, that
contain data. I have a 5th message that contains no data but closes the
group on the input queue. I need to replicate the order of items from the
input queue to the order on the output queue because the consumer of the
output queue needs to know when the group is finished in order to flush the
output file to the system. Here is the scenario I was describing explained
a bit clearer, (I hope since I included a bit of source)
Step 1: messages 1 through 3 are processed normally and put on the output
queue.
Step 2: message 4 begins processing, then message 5, the "close group"
message gets processed simultaneously. Message 5 will most likely be
inserted on the output queue before message 4 since there will be no
processing of the closing message. In other words, input message 4 will not
be in the group in the output queue since it will already be closed. This
will cause the last record of the input queue to be lost on the output
queue. Is there any way to avoid doing this. I've included the code I'm
envisioning to do the duplication from one queue to the other.
//read from input queue(passed to the MDB)
TextMessage text = (TextMessage)message;
String groupId = message.getStringProperty("JMSXGroupID");
int sequenceNumber = message.getIntProperty("JMSXGroupSeq");
String result = null;
//call POJO processing that integrates multiple systems, will probably take
around 5-10 seconds to process
if(sequenceNumber != 0){
result = pojo.process(text.getText());
}
//now lookup our output queue and send the message. Hopefully in order
Mesasge message = session.createTextMessage(result);
message.setStringProperty("JMSXGroupID", groupId);
message.setIntProperty("JMSXGroupSeq", sequenceNumber );
producer.send(message);
As you can see from the code above, message 5 will always beat message 4
onto the queue if they're received simultaneously by both nodes. How can I
get around loosing the last "real" message?
Thanks,
Todd
James.Strachan wrote:
>
> On 4/12/07, tnine <[EMAIL PROTECTED]> wrote:
>>
>> Excellent! Thank you for the help. I may have a race condition that
>> I've
>> thought of that may cause problems, but this may be taken care of my the
>> grouping. If you could give me some advice, I would really appreciate
>> it.
>>
>> It pertains to the grid processing and the nth and n-1 messages in a
>> group
>> in a 2 node or greater environment. Lets say we have n messages in the
>> group, where the last message (n) sets the sequence number to 0 to close
>> the
>> group. When my n-1 message is consumed off the input queue, it will take
>> a
>> few seconds to process. The nth message is to signal the group is
>> closed,
>> so the receiving MDB will simply push a new message with the sequence 0
>> on
>> to the output queue. The message with 0 will be on the output queue
>> before
>> the n-1 sequence is pushed. Won't my n-1 message get excluded from the
>> group since its inserted after 0?
>
> I don't quite follow. Note that the sequence number is not used to
> reorder the messages in the group. The zero is only used to close a
> group. (To be honest there's no real need to close a group since we're
> using hash buckets under the covers by default so there's little cost
> to keeping a group open).
>
>
>> This is an implementation detail, but an
>> important one. Below is an ASCII diagram of my queues and MDB
>>
>> Standalone file reader--->Input Queue (jms/StatInput)--> MDB >-- Output
>> Queue(jms/StatOutput) --> Consumer
>>
>> As you can see, I have 2 queues for round trip communication, will this
>> cause the sequences issues I described above? If so, do you have any
>> ideas
>> on how I can get around them?
>
> Each queue is separate and wil be ordered based on the order in which
> messages (within a group) are published; so it all sounds fine to me
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
>
--
View this message in context:
http://www.nabble.com/Input-queue-and-output-queue-grouping-and-sequences-question-tf3554541s2354.html#a9961009
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
