[
https://issues.apache.org/jira/browse/KAFKA-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16205108#comment-16205108
]
Ted Yu commented on KAFKA-6047:
-------------------------------
Does this require a KIP ?
> Allow retries configuration for InternalTopicManager
> ----------------------------------------------------
>
> Key: KAFKA-6047
> URL: https://issues.apache.org/jira/browse/KAFKA-6047
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.11.0.0
> Reporter: Dmitry Vsekhvalnov
> Priority: Minor
>
> There is hardcoded number of retries when kafka-streams attempts to create
> internal topics.
> *InternalTopicManager.MAX_TOPIC_READY_TRY*
> Which is not resilient in some scenarios. Consider setup where replication
> factor for internal streams topics == number of brokers. (RF=3 and x3 kafka
> brokers). When any of brokers dies kafka-streams can shutdown before broker
> is resurrected with approximate log:
> {code}
> [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager]
> [Could not create internal topics: Found only 2 brokers, but replication
> factor is 3. Decrease replication factor for internal topics via
> StreamsConfig parameter "replication.factor" or add more brokers to your
> cluster. Retry #2]
> [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager]
> [Could not create internal topics: Found only 2 brokers, but replication
> factor is 3. Decrease replication factor for internal topics via
> StreamsConfig parameter "replication.factor" or add more brokers to your
> cluster. Retry #3]
> [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager]
> [Could not create internal topics: Found only 2 brokers, but replication
> factor is 3. Decrease replication factor for internal topics via
> StreamsConfig parameter "replication.factor" or add more brokers to your
> cluster. Retry #4]
> [INFO ] [org.apache.kafka.streams.processor.internals.StreamThread]
> [stream-thread [Shutting down]
> {code}
> Would be nice if kafka-streams provides configuration for
> InternalTopicManager retries and ideally for retry backoff strategy. To have
> possibility for resilience tuning. Possibly can re-use corresponding producer
> configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
