[
https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994268#comment-15994268
]
Ewen Cheslack-Postava commented on KAFKA-4667:
----------------------------------------------
[~rhauch] Discussed some of these issues offline, but there are some other
refinements here too:
* based on KIP-115, default replication factor of 3 but override it in our demo
configs to make running on a single local node easy is probably the least
likely to cause unexpected problems
* partitions = 1 is only really true for config topic where ordering between
differing keys is required. For status and offsets topics you can go higher,
and Confluent recommends doing so:
http://docs.confluent.io/current/connect/userguide.html#distributed-mode
* Not sure if we should fiddle with unclean leader election defaults. But maybe
it is ok to set this to false and tell users if they want the unsafe version
they need to set up the topic themselves before starting Connect? Not sure what
the right tradeoff of respecting users' existing settings vs encouraging better
behavior is. This might even differ between the three topics -- status topics
are way less critical than config topic.
> Connect should create internal topics
> -------------------------------------
>
> Key: KAFKA-4667
> URL: https://issues.apache.org/jira/browse/KAFKA-4667
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Emanuele Cesena
> Priority: Critical
> Fix For: 0.11.0.0
>
>
> I'm reporting this as an issue but in fact it requires more investigation
> (which unfortunately I'm not able to perform at this time).
> Repro steps:
> - configure Kafka for consistency, for example:
> default.replication.factor=3
> min.insync.replicas=2
> unclean.leader.election.enable=false
> - run Connect for the first time, which should create its internal topics
> I believe these topics are created with the broker's default, in particular:
> min.insync.replicas=2
> unclean.leader.election.enable=false
> but connect doesn't produce with acks=all, which in turn may cause the
> cluster to go in a bad state (see, e.g.,
> https://issues.apache.org/jira/browse/KAFKA-4666).
> Solution would be to force availability mode, i.e. force:
> unclean.leader.election.enable=true
> when creating the connect topics, or viceversa detect availability vs
> consistency mode and turn acks=all if needed.
> I assume the same happens with other kafka-based services such as streams.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
