Before we get into the details of your use-case I have a few questions... 1) Have you conducted benchmark tests and demonstrated conclusively that you cannot meet your performance goals with a single broker (or HA pair of brokers)? If so, can you share any details about your testing and the results? 2) Have you read about and understood the performance considerations for clustering [1]?
Justin [1] https://activemq.apache.org/components/artemis/documentation/latest/clusters.html#performance-considerations On Fri, Apr 11, 2025 at 9:39 AM Luchi Mirko <mirko.lu...@siav.it> wrote: > Hi, > > we are planning to adopt Artemis in our software solution as substitute of > the current message broker (RabbitMQ). > We are especially interested in 2 features it offers: > > * > message grouping > * > automatic rebalancing of message groups. > > We need message grouping because our consumers must ensure that certain > groups of messages are processed serially. > We need rebalancing because we deploy our solution in an elastic > environment and we want to be able to scale the application horizontally. > > Ideally, we would like to use a cluster of Artemis brokers for load > balancing, so we read the clustered grouping chapter of the documentation ( > https://activemq.apache.org/components/artemis/documentation/latest/message-grouping.html#clustered-grouping) > and we are aware of the potential pitfalls in this choice. > > To simplify let's say we have a single address to which we publish > messages (each message with groupId set), and multiple shared durable > subcriptions that load balance consumption of those messages. > Given that we will scale the application horizontally, we can assume that > each node of the Artemis cluster will eventually have a queue bound to that > address with some groupIds sticked to it. > We know that we must make sure that each of these queues must have at > least one consumer attached, or we might find ourselves in a situation > where a message is routed (because of message grouping) to a queue that has > no consumers attached, and hence that message will never get a chance to be > consumed. > > We can make sure (properly tuning autoscaling of our consumer pods) that > the minimum number of consumers is always equals or greater than the number > of broker nodes (a.k.a. number of queues), but even doing so it might > happen that multiple consumers register on the same queue on the same > broker node, leaving therefore queues without consumers. > Is there a way to specify some kind of policy by which client should > preferably first connect to queues without consumers, so that consumers are > distributed evenly and we can guarantee that no queue will remain without > consumer (given of course that we make sure to have at least a number of > consumers equal to the broker nodes)? > > Thanks > > >
