Hey Dan, I think this is a very useful idea. Two questions:
SVV1: Do you think we need the feature flag at all? I know that not having this flag may technically render the KIP unnecessary (however it may still be useful to discuss this topic and create a concensus). As you wrote in the KIP, we may be able to look up the target and source topics and if we can do this, we can probably detect if the replication is one-way or prefixless (identity). So that means we don't need this flag to control when we want to use this. Then it is really just there to act as something that can turn the feature on and off if needed, but I'm not really sure if there is a great risk in just enabling this by default. If we really just turn back the B -> A checkpoints and save them in the A -> B, then maybe it's not too risky and users would get this immediately by just upgrading. SVV2: You write that we need DefaultReplicationPolicy to use this feature, but most of the functionality is there on interface level in ReplicationPolicy. Is there anything that is missing from there and if so, what do you think about pulling it into the interface? If this improvement only works with the default replication policy, then it's somewhat limiting as users may have their own policy for various reasons, but if we can make it work on the interface level, then we could provide this feature to everyone. Of course there can be replication policies like the identity one that by design disallows this feature, but for that, see my previous point. Best, Viktor On Fri, Oct 18, 2024 at 3:30 PM Dániel Urbán <urb.dani...@gmail.com> wrote: > Hi everyone, > > I'd like to start the discussion on KIP-1098: Reverse Checkpointing ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1098%3A+Reverse+Checkpointing > ) > which aims to minimize message reprocessing for consumers in failbacks. > > TIA, > Daniel >
