That's exactly right. We've been talking about this internally at LinkedIn, and how to solve it. I think the best option would be to have the broker throw an error on offset commits until there are enough brokers to fulfill the configured RF.
We've seen this several times now when bootstrapping test and development clusters. You can work around it by making sure all consumers are shut down until you have all the brokers started. But that's not a good long term solution for this problem. -Todd > On Aug 5, 2015, at 6:28 PM, James Cheng <jch...@tivo.com> wrote: > > Hi, > > My kafka cluster has a __consumer_offsets topic with 50 partitions (the > default for offsets.topic.num.partitions) but with a replication factor of > just 1 (the default for offsets.topic.replication.factor should be 3). > > From the docs http://kafka.apache.org/documentation.html: > > offsets.topic.replication.factor 3 The replication factor for the > offset commit topic. A higher setting (e.g., three or four) is recommended in > order to ensure higher availability. If the offsets topic is created when > fewer brokers than the replication factor then the offsets topic will be > created with fewer replicas. > > > I'm guessing there's a typo there? I'm guessing it should be: > > If the offsets topic is created when fewer brokers than the replication > factor [are active], then the offsets topic will be created with fewer > replicas. > > Or something along those lines? > > Thanks, > -James > >
