smjn commented on PR #18712: URL: https://github.com/apache/kafka/pull/18712#issuecomment-2616180735
> Thanks for this! I am still catching up, so apologies if some of the questions don't make too much sense. I left a couple of comments here and there, but I had a more generic question. Is there something preventing a retried request to result in multiple tombstones? If not, is this problematic? If you mean not being able to write to the log by the replica manager, then the coordinator runtime manages that. If it is unable to commit the tombstone - the CoordinatorRuntime.scheduleWriteOperation should eventually fail. The retry logic is encapsulated in the PersisterStateManager which only retries on certain well known error codes. These calls are not meant to be made directly via the client. So consider the following points in the call flow - CoordinatorRuntime (guarantee that record is written or failed) - KafkaApis (response might be lost) -> but the persister state manager only retries on well known errors and NOT on TimeoutException. Lets consider the worst case - If multiple tombstones do get written such that - ``` INITIAL_TOMBSTONE_share_part_key_1 valid_record_1_share_part_key_1 valid_record_2_share_part_key_2 BAD_RETRY_NEW_TOMBSTONE_share_part_key_1 ``` On replay of `BAD_RETRY_NEW_TOMBSTONE_share_part_key_1`. The state of `share_part_key_1` will be cleared in the ShareCoordinatorShard. The state is stored in TimelineHashMaps encapsulated in ShareCoordinatorShard. Later, when a read call happens this lost information will not be returned and as a result the ShareParitition will re-acquire the records and resend them. This by nature falls within the realm of at least once semantics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
