Yeah that makes sense, but what if we do need to change the number of partitions? What if we need to reduce it?
On Tue, Jan 8, 2013 at 12:42 PM, Jun Rao <jun...@gmail.com> wrote: > If you don't have a lot of topics, one thing you can do is to > over-partition a topic. > > Also, in 0.7, # of partitions grows with brokers. This is going to change > in 0.8, in which # of partitions is specified at topic creation time and > won't change as brokers change. One needs to use an admin DDL to change # > of partitions. > > Thanks, > > Jun > > On Mon, Jan 7, 2013 at 10:23 PM, David Ross <dyr...@klout.com> wrote: > > > Hello, > > > > We have found that, for our application, having a number of total > > partitions as a multiple of the number of consumer hosts is beneficial. > > Because of this, whenever we add or remove consumer hosts, we have to > > change the number of partitions in the server config. > > > > What are best practices for changing the number of partitions? It seems > > like adding partitions is fine but removing partitions would result in > data > > loss - am I right? Is that avoidable? Is it preferable to bring in new > > servers with new partitions? Anything else I should keep in mind on this > > issue? > > > > > > Thanks! > > > > David > > >
