[
https://issues.apache.org/jira/browse/KAFKA-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419471#comment-15419471
]
Grant Henke commented on KAFKA-3959:
------------------------------------
I would like to present an alternative option. This problem exists with any
topic created using default.replication.factor > 1 as well. That prevents using
2 or 3 as the configuration default because we want to support single nodes
clusters without changing the defaults.
Instead of preventing topics from being created with a low replication factor
(unless min.isr is set). Instead it would be really nice if we tracked a
"target replication factor" in the topic metadata. This is an improvement over
assuming the target replication factor based on the actual replicas as is done
today and can actually result in a more accurate under replicated count.
This change would also help support any ability to automatically maintain the
desired replication factor as nodes are started, stopped, etc. Some related
KIPs for that are:
* [KIP-73 Replication
Quotas|https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas]
* [KIP-46: Self Healing
Kafka|https://cwiki.apache.org/confluence/display/KAFKA/KIP-46%3A+Self+Healing+Kafka]
Would that be a viable option?
> __consumer_offsets wrong number of replicas at startup
> ------------------------------------------------------
>
> Key: KAFKA-3959
> URL: https://issues.apache.org/jira/browse/KAFKA-3959
> Project: Kafka
> Issue Type: Bug
> Components: consumer, offset manager, replication
> Affects Versions: 0.9.0.1, 0.10.0.0
> Environment: Brokers of 3 kafka nodes running Red Hat Enterprise
> Linux Server release 7.2 (Maipo)
> Reporter: Alban Hurtaud
>
> When creating a stack of 3 kafka brokers, the consumer is starting faster
> than kafka nodes and when trying to read a topic, only one kafka node is
> available.
> So the __consumer_offsets is created with a replication factor set to 1
> (instead of configured 3) :
> offsets.topic.replication.factor=3
> default.replication.factor=3
> min.insync.replicas=2
> Then, other kafka nodes go up and we have exceptions because the replicas #
> for __consumer_offsets is 1 and min insync is 2. So exceptions are thrown.
> What I missed is : Why the __consumer_offsets is created with replication to
> 1 (when 1 broker is running) whereas in server.properties it is set to 3 ?
> To reproduce :
> - Prepare 3 kafka nodes with the 3 lines above added to servers.properties.
> - Run one kafka,
> - Run one consumer (the __consumer_offsets is created with replicas =1)
> - Run 2 more kafka nodes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
