Re: Kafka Connect - offset.storage.topic reuse across clusters

2022-03-30 Thread Chris Egerton
Connectors overwriting each other's offsets is the primary concern. If you have a guarantee that there will only ever be one connector with a given name running at once on any of the Connect clusters that use the same offsets topic, and you want offsets to be shared for all source connectors on any

Re: Kafka Connect - offset.storage.topic reuse across clusters

2022-03-30 Thread Jordan Wyatt
Hi Robin, I'm interested in a use case in which I need to be able to have a connect cluster fail, and then bring up a new cluster with the same offset topics and connectors. By new cluster I mean a cluster with a new `group.id`. I am aware I could just use the same group id as before but I would l

Re: Kafka Connect - offset.storage.topic reuse across clusters

2022-03-30 Thread Robin Moffatt
Hi Jordan, Is there a good reason for wanting to do this? I can think of multiple reasons why you shouldn't do this even if technically it works in some cases. Or it's just curiosity as to whether you can/should? thanks, Robin. -- Robin Moffatt | Principal Developer Advocate | ro...@confluent