Hi, Thanks for the reply. My earlier distinction was sloppy, I ought to have written something like 'partially connected mesh' vs. a complete graph. I was trying to distinguish between a topology where nodes are only connected to immediate neighbors, and therefore may always be required to travel across multiple brokers to get to a particular destination.
In this particular setup all the brokers will have the same capabilities and will be connected to the same network, so I think I'll go ahead with these scheme. I appreciate the feedback! Jim On Sun, Apr 26, 2015 at 9:10 AM Tim Bain <tb...@alumni.duke.edu> wrote: > Your understanding of this is good and you've got the right concepts here. > The only two reasons I can think of that you shouldn't use a mesh (which I > think is the same as a complete graph, though you drew a distinction > between the two in your email so maybe there's a difference I'm not > understanding) is if you have non-homogeneous network links where the most > efficient routing between two brokers isn't necessarily the path "directly" > (from a network-of-brokers standpoint, not from a network standpoint) > between the two, or if you don't want to have to specify all of the other > clusters so you'd use a hub-and-spoke topology so you only have to specify > the hub node in your config. > > On Fri, Mar 13, 2015 at 12:21 AM, James A. Robinson < > jim.robin...@gmail.com> > wrote: > > > Hi folks, > > > > I joined this list because we're starting to look into setting up > > ActiveMQ for our developers. My preference at this time is to set > > up a highly reliable system that doesn't have single points of > > failure. > > > > Due to the concern about single points of failure I didn't want to > > introduce a shared persistence layer. That's why I've been looking > > at replicated LevelDB. > > > > After spending some time reading up on ActiveMQ my first inclination > > has been to propose setting up 3-node replicated LevelDB clusters, > > and to build a network of brokers out of sets of those 3-node > > clusters. > > > > E.g., start with two clusters, and then keep adding on as we see > > the need in terms of capacity. > > > > [amq-1a, amq-1b, amq-1c] > > [amq-2a, amq-2b, amq-2c] > > ... > > [amq-9a, amq-9b, amq-9c] > > ... > > > > I'm not sure what the best option is w/re to the network of broker > > topology. I could see mesh working, but my first instinct would > > be to set up a complete graph, and have each broker cluster configured > > with a uni-directional network connector to every other cluster > > using the masterslave scheme. > > > > I am wondering if this is a naive approach, and if so why it wouldn't > > be a good idea. > > > > If it is reasonable, I'm still trying to fully understand how one > > deals with forwarding messages from brokers w/o consumers to those > > brokers w/ consumers. The page > > > > http://activemq.apache.org/networks-of-brokers.html > > > > discusses this topic and mentions rebalanceClusterClients, and it > > isn't clear to me how that option will help if the clusters and > > clients are fairly static: > > > > rebalanceClusterClients: > > if true, connected clients will be asked to rebalance across a > > cluster of brokers when a new broker joins the network of brokers > > (note: priorityBackup=true can override) > > > > Reading the list of options available for transport connectors, it > > seems reasonable to configure the transport connector with: > > > > updateClusterClients="true" > > rebalanceClusterClients="true" > > updateClusterClientsOnRemove="true" > > > > But if one has mostly static clients and we aren't frequently > > bringing broker clusters up and down, I would assume there would > > still be a problem of having messages on a broker that doesn't have > > consumers attached. If I'm wrong I'd like to understand what I'm > > missing. > > > > The network-of-brokers page, after mentioning rebalanceClusterClients > > as one solution, continues the discussion and states that: > > > > Another, is to allow replay of messages back to the origin as > > there is no local consumer on that broker. > > > > There is a destination policy that allows this behavior for queues > > by configuring a conditionalNetworkBridgeFilterFactory with > > replayWhenNoConsumers=true. The conditionalNetworkBridgeFilterFactory > > provides an optional replayDelay based on the broker-in time. > > > > N.B.: When using replayWhenNoConsumers=true for versions < 5.9, > > it is necessary to also disable the cursors duplicate detection > > using enableAudit=false as the cursor could mark the replayed > > messages as duplicates (depending on the time window between > > playing and replaying these messages over the network bridge). > > The problem is fully explained in this blog post. > > > > And I see in that ticket AMQ-4607 says this about the network > > connector options: > > > > networkTTL sets both consumerTTL and messageTTL a value of -1 > > denotes infinite hops. > > In a mesh, use consumerTTL=1, messageTTL=-1 to allow consumers to > > bounce around the mesh and messages to follow demand. > > In a linear topology use networkTTL == length of network (as before) > > > > So would it be reasonable to set up a complete graph, with each > > 3-node cluster configured with a uni-directional network connector > > to every other 3-node cluster using masterslave, and to set > > > > consumerTTL="1" > > messageTTL="-1" > > decreaseNetworkConsumerPriority="true" > > > > and a networkBridgeFilterFactory/conditionalNetworkBridgeFilterFactory > > with > > > > replayWhenNoConsumers="true" > > > > I'd appreciate any input from the folks here, including any pointers > > on any more material I should read. > > > > Jim > > >
