If you don't have a lot of topics, one thing you can do is to over-partition a topic.
Also, in 0.7, # of partitions grows with brokers. This is going to change in 0.8, in which # of partitions is specified at topic creation time and won't change as brokers change. One needs to use an admin DDL to change # of partitions. Thanks, Jun On Mon, Jan 7, 2013 at 10:23 PM, David Ross <dyr...@klout.com> wrote: > Hello, > > We have found that, for our application, having a number of total > partitions as a multiple of the number of consumer hosts is beneficial. > Because of this, whenever we add or remove consumer hosts, we have to > change the number of partitions in the server config. > > What are best practices for changing the number of partitions? It seems > like adding partitions is fine but removing partitions would result in data > loss - am I right? Is that avoidable? Is it preferable to bring in new > servers with new partitions? Anything else I should keep in mind on this > issue? > > > Thanks! > > David >
