[jira] [Created] (KAFKA-20091) Fix inconsistency in time-based retention checks between remote and local segment deletion logic.

[fujian (Jira)]

fujian created KAFKA-20091:
------------------------------

             Summary: Fix inconsistency in time-based retention checks between 
remote and local segment deletion logic. 
                 Key: KAFKA-20091
                 URL: https://issues.apache.org/jira/browse/KAFKA-20091
             Project: Kafka
          Issue Type: Bug
          Components: Tiered-Storage
            Reporter: fujian
            Assignee: fujian


When currentTime- largestTimestamp of segment = retentionMs, the behavior is 
different:
 # Local Segment delete: *Not delete*

org.apache.kafka.storage.internals.log.UnifiedLog#deleteRetentionMsBreachedSegments:
   boolean delete = startMs - segment.largestTimestamp() > retentionMs;

  # Remote segement delete: *Delete*

org.apache.kafka.server.log.remote.storage.RemoteLogManager.RLMExpirationTask.RemoteLogRetentionHandler#isSegmentBreachedByRetentionTime
shouldDeleteSegment = metadata.maxTimestampMs() <= 
retentionTimeData.get().cleanupUntilMs;

//cleanupUntilMs is time.milliseconds() - retentionMs;
private Optional<RetentionTimeData> buildRetentionTimeData(long retentionMs) \{
    long cleanupUntilMs = time.milliseconds() - retentionMs;
    return retentionMs > -1 && cleanupUntilMs >= 0
            ? Optional.of(new RetentionTimeData(retentionMs, cleanupUntilMs))
            : Optional.empty();
}{{}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)