Jun Rao created KAFKA-4529:
------------------------------
Summary: tombstone may be removed earlier than it should
Key: KAFKA-4529
URL: https://issues.apache.org/jira/browse/KAFKA-4529
Project: Kafka
Issue Type: Bug
Affects Versions: 0.10.1.0
Reporter: Jun Rao
Fix For: 0.10.1.1
As part of KIP-33, we introduced a regression on how tombstone is removed in a
compacted topic. We want to delay the removal of a tombstone to avoid the case
that a reader first reads a non-tombstone message on a key and then doesn't see
the tombstone for the key because it's deleted too quickly. So, a tombstone is
supposed to only be removed from a compacted topic after the tombstone is part
of the cleaned portion of the log after delete.retention.ms.
Before KIP-33, deleteHorizonMs in LogCleaner is calculated based on the last
modified time, which is monotonically increasing from old to new segments. With
KIP-33, deleteHorizonMs is calculated based on the message timestamp, which is
not necessarily monotonically increasing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
