That's a good question. Off the top of my head I don't remember any fundamentally good reason why we don't allow it - apart from: - broker registration paths are ephemeral so topic creation cannot succeed when there are insufficient brokers available - it may be confusing to some users to successfully create a topic and allow its partition replicas to be assigned to brokers that are unavailable
One can possibly argue that topic creation should not be permitted with _any_ unavailable brokers since extended unavailability could result in imbalanced partition distribution (when the unavailable brokers are restored). Anyway, unless I'm missing something obvious I think your point is worth discussing further in a jira. Joel On Tue, Oct 15, 2013 at 1:47 PM, Jason Rosenberg <j...@squareup.com> wrote: > Is there a fundamental reason for not allowing creation of new topics while > in an under-replicated state? For systems that use automatic topic > creation, it seems like losing a node in this case is akin to the cluster > being unavailable, if one of the nodes goes down, etc. > > > On Tue, Oct 15, 2013 at 1:25 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > >> Steve - that's right. I think Monika wanted clarification on what >> would happen if replication factor is two and only one broker is >> available. In that case, you won't be able to create new topics with >> replication factor two (you should see an AdministrationException >> saying the replication factor is larger than available brokers). >> >> However, you can send messages to available partitions of topics that >> have already been created - because the ISR would shrink to only one >> replica for those topics - although the cluster would be in an >> under-replicated state. This is covered in the documentation >> (http://kafka.apache.org/documentation.html#replication) under the >> discussion about ISR. >> >> Thanks, >> >> Joel >> >> On Tue, Oct 15, 2013 at 10:19 AM, Steve Morin <st...@stevemorin.com> >> wrote: >> > If you have a double broker failure with replication factor of 2 and only >> > have 2 brokers in the cluster. Wouldn't every partition be not >> available? >> > >> > >> > On Tue, Oct 15, 2013 at 8:48 AM, Jun Rao <jun...@gmail.com> wrote: >> > >> >> If you have double broker failures with a replication factor of 2, some >> >> partitions will not be available. When one of the brokers comes back, >> the >> >> partition is made available again (there is potential data loss), but >> in an >> >> under replicated mode. After the second broker comes back, it will >> catch up >> >> from the other replica and the partition will eventually be fully >> >> replicated. There is no need to change the replication factor during >> this >> >> process. >> >> >> >> As for ZK, you can always use the full connection string. ZK will pick >> live >> >> servers to establish connections. >> >> >> >> Thanks, >> >> >> >> Jun >> >> >> >> >> >> On Tue, Oct 15, 2013 at 3:46 AM, Monika Garg <gargmon...@gmail.com> >> wrote: >> >> >> >> > I have 2 nodes kafka cluster with default.replication.factor=2,is set >> in >> >> > server.properties file. >> >> > >> >> > I removed one node-in removing that node,I killed Kafka >> process,removed >> >> all >> >> > the kafka-logs and bundle from that node. >> >> > >> >> > Then I stopped my remaining running node in the cluster and started >> >> > again(default.replication.factor is still set to 2 in this node >> >> > server.properties file). >> >> > I was expecting some error/exception as now I don't have two nodes in >> my >> >> > cluster.But I didn't get any error/exception and my node successfully >> >> > started and I am able to create topics on it. >> >> > >> >> > So should the "default.replication.factor" be updated from >> >> > "default.replication.factor=2" to "default.replication.factor=1" , in >> the >> >> > remaining running node? >> >> > >> >> > Similarly if there are two external zookeeper >> >> > nodes(zookeeper.connect=host1:port1,host2:port1) in my cluster and >> now I >> >> > have removed one zookeeper node(host1:port1) from the cluster,So >> should >> >> the >> >> > property "zookeeper.connect" be updated from >> >> > (zookeeper.connect=host1:port1,host2:port1) to >> >> > (zookeeper.connect=host2:port1)? >> >> > >> >> > -- >> >> > *Moniii* >> >> > >> >> >>
