Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread James Strachan
On 05/12/2007, Marc Zampetti <[EMAIL PROTECTED]> wrote: > > James, > > AMQ-816 sounds exactly like what I need. Then I could configured all of > these brokers as basically stand-alone with the consumers connected to a > sub-set or all of them, depending upon what I need. Do you know if AMQ-816 > is

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread Marc Zampetti
James, AMQ-816 sounds exactly like what I need. Then I could configured all of these brokers as basically stand-alone with the consumers connected to a sub-set or all of them, depending upon what I need. Do you know if AMQ-816 is going to be implemented, and if so, any ideas when? Marc James.S

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread James Strachan
On 05/12/2007, ttmdev <[EMAIL PROTECTED]> wrote: > > Given that Marc's producers will be sending non-persistent messages, wouldn't > a shared - as opposed to pure - master/slave configuration provide > redundancy at the broker level and do so w/no extra overhead? Shared File System & Shared Databa

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread ttmdev
Given that Marc's producers will be sending non-persistent messages, wouldn't a shared - as opposed to pure - master/slave configuration provide redundancy at the broker level and do so w/no extra overhead? My thinking was that if the master were to fail, you can its clients failover to the slave

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread James Strachan
On 05/12/2007, Marc Zampetti <[EMAIL PROTECTED]> wrote: > James, > > Yes, it sounds like the JEDI thing and the partitioning approach is what I > need. And yes, I'm talking queues for the most part. For the partitioning, > is that something that AMQ, or are you talking about me having a layer in >

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread Marc Zampetti
James, Yes, it sounds like the JEDI thing and the partitioning approach is what I need. And yes, I'm talking queues for the most part. For the partitioning, is that something that AMQ, or are you talking about me having a layer in front of AMQ that would do this. In this case, instead of having

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread James Strachan
On 05/12/2007, Marc Zampetti <[EMAIL PROTECTED]> wrote: > James, > > You are right about the HA comment I originally made. I was referring to > fact that I'm not looking for persistent messages. But I am concerned about > what happens when a broker fails and being able to recover from that > quickl

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread Marc Zampetti
James, You are right about the HA comment I originally made. I was referring to fact that I'm not looking for persistent messages. But I am concerned about what happens when a broker fails and being able to recover from that quickly, even if that means losing messages. I understand the point abo

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread James Strachan
BTW I thought you said previously that you were not that concerned with persistence / HA? Before we can really know the right ActiveMQ architecture to solve your problem we need a few more details such as how many producers, consumers, do you want queues or topics and exactly what the traffic shap

Re: Questions on Network of Brokers and high message rates

2007-12-05 Thread ttmdev
The master/slave configuration for AMQ should address your HA concerns. In your case, I think a shared file system or jdbc master/slave would be the way to go. Have you used the maven 2 or jmeter performance tests for benchmarking? http://activemq.apache.org/jmeter-performance-tests.html http

Re: Questions on Network of Brokers and high message rates

2007-12-04 Thread Marc Zampetti
Joe, Thanks for the suggestion. But in this case, it won't work since the routing criteria are much too fine-grained. Basically, each of the 6 million messages will each have a unique id. Then some subset, say 1 million messages, will need to be routed to a process for special processing. Basical

Re: Questions on Network of Brokers and high message rates

2007-12-04 Thread ttmdev
Re the second part of your post. If Camel is not an option, then what about a composite queue in combination with selectors? For example, in the snippet below, Q.FOO gets a subset of the message stream being sent to Q.BLAST, while Q.BAR gets the entire stream.