C0urante commented on PR #11818: URL: https://github.com/apache/kafka/pull/11818#issuecomment-1367408152
Thanks for this fix, @emilnkrastev. I want to make sure I understand the goal here before commenting on the actual code changes (although I've made sure to read through them carefully). It looks like incorrect updates to the `partitionStates` field (or its contents) might result in the starvation of offset syncs for some topic partitions. The task will believe that it has performed syncs that it hasn't, and as a result, may never see that the lag between the (true) last-synced offset and the offset for the last-replicated record has exceeded the value for the user-configured `offset.lag.max` property. However, it looks like KAFKA-12468 and possibly KAFKA-12558 describe a different problem: it's not just a matter of offset syncs being delayed for longer than expected or even indefinitely; instead, the offset syncs that do make it to the downstream cluster actually contain incorrect values, which in some extreme cases can result in things like negative consumer lag. I don't see how this PR would address an issue with correctness in the actual offsets used for syncs; instead, it seems like the value here is that it will allow offset syncs to take place in cases where they should be but are currently not. Is that a fair assessment of the goal here? If so, I'm happy to review/merge a fix with that goal, and if not, would you mind shedding some light on what the goal is and where I might be missing something? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
