> > > > I think there's a limit to how many redelivery attempts you're willing to > take before to send the message to the DLQ, which I think would cover most > scenarios when that would happen in the wild. (You could always construct > an arbitrarily bad failure case, but the odds of actually seeing it in the > real world get vanishingly small as it gets uglier.) > > That is true. Right now our DLQ redelivery policy is 5 ... so we would hit the limit of 5 and the secondar messages would get re-executed.
> > So if we have 10 servers, each thread has ten sessions. and if prefetch > is > > 1 then that means I prefetch 10 total messages. If each message takes 30 > > seconds to execute that thread will take a while to handle all ten. > > This leads to significant latency. > > > > If I'm understanding correctly, you've got a single client consuming one > message at a time while consuming from N brokers that are presumably not > networked (otherwise why would you connect to more than one of them)? > Why? (Among other things, why not just network the brokers and simplify > your use-case?) > > I had thought about this initially but ruled out running a network of brokers for a number of reasons. 1. I wanted to keep things simple so I could port to another queue system in the future. 2. Our system is already 'ridiculously parallelizable' so just splitting the brokers up works easily enough. 3. I thought my initial implementation would be fine :) 4. The documentation for network of brokers wasn't really all there and I had a lot of residual questions and just assumed that this functionality wasn't really all there. I'm still kind of unclear how this works. For example I would want sharded and replicated brokers where we had 1/Nth of our data in one shard but that shard also has a primary and backup replica. We basically get this now as soon as I turn on replication but then again I *really* understand the topology and how it works. With the current system/documentation I'm unclear if this is even possible. Also, the mentions of JXTA make me think the documentation hasn't been updated in a decade ;) Kevin -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts>
