Hi All, I'm in a sub-optimal situation whereby I have some Kafka Streams apps deployed to production, but the default replication factor set on the brokers was 1 when they were first deployed. As such, any state store changelog topics, and re-partition topics therefore have RF 1 also.
I'm familiar with the bin/kafka-reassign-partitions.sh tool and how to use that. I've since also found the streams replication.factor (StreamsConfig.REPLICATION_FACTOR_CONFIG) setting that can be supplied in the streams apps config. My questions are: 1. Will simply changing the value of REPLICATION_FACTOR_CONFIG and re-deploying have any effect on already-created internal topics? 2. Conversely, should I just change the RF of the internal topics using the re-assign-partitions tool? Is that safe to do whilst the apps are still running? 3. (not Streams-specific) If a broker that was the leader of partition(s) for a topic with RF 1 died (i.e. no replicas), and was non-recoverable - would the only way to restore service to those partitions be to delete the topic and re-create? (with data loss, understandably). I couldn't seem to achieve this using preferred/unclean leader election, nor using the re-assign-partitions tool, since that requires all brokers to be healthy. Many thanks in advance for any answers, Marcus