Hi, In our cluster (across many), rarely I see a deleted key reappearing during consume. This happens in kafka 3.2 and also in 3.9 (2 versions used in various cluster)
Example: Key:abc (offset 100) - sent at T1 Tombstone for key:abc - sent at T1 + 2 days (Tombstone delete.retention.ms is 1 day) Consume at T1 + 4 days (rehydration - consume from beginning) - No key Consume at T1 + 6 days (rehydration - consume from beginning) - key:abc appears with same offset(100) These number of days varies but I get this issue approximately 1% of time (not consistent) Anybody faced this issue and any solution recommended. My topic config is below Partition=10 Replication-factor=5 min.insync.replicas=2 cleanup.policy=compact segment.bytes=15728640 min.compaction.lag.ms=900000 max.compaction.lag.ms=1800000 delete.retention.ms=86400000 retention.ms=259200000 message.timestamp.type=CreateTime Regards, Nanda
