Hi Andrew,

Thanks for the speedy reply, so do I just need to do this for partition 49? 
What about partitions 0-48, will these be covered by reassigning partition 49. 

Do I need to do this for the __transaction_state topics too?

Many thanks,
Chris

> On 16 Dec 2022, at 4:17 pm, Andrew Grant <agr...@confluent.io.invalid> wrote:
> 
> Hey Chris,
> I think you should be able to use the reassignment tool to add replicas.
> You should be able to do something similar to migrate the partitions away
> from the old brokers and onto the new ones and also increase the
> replication factor at the same time. I tested just increasing the
> replication factor with the following commands:
> 
> kafka % ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic
> __consumer_offsets --describe | grep 'Partition: 49'
> Topic: __consumer_offsets Partition: 49 Leader: 1 Replicas: 1 Isr: 1
> Offline:
> 
> kafka % cat reassignment.json
> {
>  "version": 1,
>  "partitions": [
>    {
>      "topic": "__consumer_offsets",
>      "partition": 49,
>      "replicas": [ 1, 0 ]
>    }
>  ]
> }
> 
> kafka % ./bin/kafka-reassign-partitions.sh --bootstrap-server
> localhost:9092 --reassignment-json-file reassignment.json --execute
> Current partition replica assignment
> 
> {"version":1,"partitions":[{"topic":"__consumer_offsets","partition":49,"replicas":[1],"log_dirs":["any"]}]}
> 
> Save this to use as the --reassignment-json-file option during rollback
> Successfully started partition reassignment for __consumer_offsets-49
> kafka % ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic
> __consumer_offsets --describe | grep 'Partition: 49'
> Topic: __consumer_offsets Partition: 49 Leader: 1 Replicas: 1,0 Isr: 1,0
> Offline:
> 
> 
> Andrew
> 
> 
>> On Fri, Dec 16, 2022 at 9:46 AM Chris Peart <ch...@peart.me.uk> wrote:
>> 
>> 
>> 
>> Hi,
>> 
>> We have a kafka production cluster that was setup with the defaults for
>> __consumer_offsets & __transaction_state topics.
>> 
>> Is there a way to increase the replication factor from 1 to 3 using the
>> kafka-reassign-partitions tool?
>> 
>> We are also replacing our 4 brokers with new brokers, this has been
>> completed so we now have an 8 broker cluster and have migrated all the
>> topics to the new brokers using the reassign tool except for the
>> __consumer_offsets & __transaction_state topics.
>> 
>> We stopped kafka on the old brokers today but all out consumers failed
>> due to the __consumer_offsets & __transaction_state topics residing on
>> the old brokers.
>> 
>> I'm thinking we should could move the __consumer_offsets &
>> __transaction_state topics to the now brokers using the reassign tool, i
>> have done this on our dev platform and all is good. If you think this is
>> good idea we can then can stop kafka on the old nodes and then work on
>> the replication factoron the new nodes?
>> 
>> The problem i have is how do we change the replication factor to 3 after
>> we migrate the __consumer_offsets & __transaction_state topics?
>> 
>> Thanks in advance.
>> 
>> Chris

Reply via email to