Mike, The biggest issue with geographically distributed systems is the latency introduced by the distances between your locations.
You'll definitely want to tune your TCP stack, including computing an adequate TCP buffer size for the bandwidth-delay product <https://en.wikipedia.org/wiki/Bandwidth-delay_product>. You may also want to tell the kernel not to change the TCP congestion window during idle periods, as I found when I investigated some undesirable behavior in http://stackoverflow.com/questions/25494929/3x-latency-observed-in-activemq-only-for-higher-wan-latencies. And you should absolutely brush up on the details of how TCP works, because understanding (or remembering, if like me it had been a while since you learned it) the intricacies of the TCP protocol can help you understand what's going on when you start having performance problems. Also, you're going to want to consider geography when making decisions about how to architect the ActiveMQ portions of your overall system (brokers, consumers, and producers). Your performance will generally be best when you can avoid having to jump across a high-latency link, so where possible you should try to localize processing to ensure that producer, consumer, and broker(s) are all in the same part of the world. You may want to use the priorityBackup feature of the failover protocol to ensure that you always connect to the nearby broker(s) whenever they are available. Also, where you need to cross a high-latency link, you'll have the best results if you can do it asynchronously (so your program can go on processing in the meantime), so ideally you'd never want a client (especially a producer) to have to cross a high-latency link to connect to a broker; if anything, make the brokers communicate across the high-latency links, so the clients can be working on the next message while the transfer is in progress. If some of your clients (and especially your producers) aren't located close to any of your brokers, you might want to consider co-locating an additional broker with those clients, so the client connection to the first broker is fast. That could be done by deploying a standalone broker that's always running, or by running an embedded broker within the client process. But be careful of the latter option; it means that when the client is stopped, all messages on the embedded broker at that time are no longer available to the network of brokers, which can lead to undesirable results. Make sure you think through all the scenarios very carefully before you choose the embedded broker option. The final thing to think about is that in this type of configuration, network partitions become a very real possibility, so you should make sure that your overall system will behave acceptably in the face of a network partition (or that it's able to detect one and stop processing until the problem is resolved). Whether this is a problem and how hard it is to resolve depends heavily on the domain of your particular problem. Tim On Aug 18, 2016 9:09 AM, "mtod" <m...@thetods.net> wrote: > Thanks Tim > > I have it working now with the masterslave option. > > Question: I'm setting up a masterslave cluster in Germany, China and other > US Based locations is there any best practices for distributed clusters? > > Mike > > > > -- > View this message in context: http://activemq.2283324.n4.nab > ble.com/Network-of-brokers-setup-error-tp4715603p4715639.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
