Hi Daniel, This would indeed greatly reduce the duplicate processing on failbacks.
Few questions: 1. Adding another offset sync store might be memory intensive. Would it make sense (if possible) to filter the topics in it based on the reverseCheckpointingTopicFilter? 2. Would it make sense to add a reverseCheckpointingGroupFilter as well, so that one can control not just the topics for reverse checkpointing but also the groups? Do I understand this correctly, that the replication flow itself must be bidirectional, but the topic replication doesn’t? If so, this seems to unlock another use case. With this change, one can more confidently fail over the consumer group to the passive cluster and back (in the context of the topic itself), without much reprocessing; I see this useful when a cluster gets busy at times. Or even have a new consumer group consume messages from the passive cluster for a while, before “failing it over” to the active cluster. Is this something that you would recommend using the feature for? Best, Vidor On 2024/10/18 13:29:32 Dániel Urbán wrote: > Hi everyone, > > I'd like to start the discussion on KIP-1098: Reverse Checkpointing ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1098%3A+Reverse+Checkpointing) > which aims to minimize message reprocessing for consumers in failbacks. > > TIA, > Daniel >
