Bubbling this up to understand if anyone else are in similar use case.
-----Original Message----- From: Koushik Chitta <kchi...@microsoft.com.INVALID> Sent: Sunday, February 23, 2020 1:35 PM To: us...@kafka.apache.org; dev@kafka.apache.org Subject: [EXTERNAL] Issue in retention with compact,delete cleanup policy Hi, I have a Topic with following config. cleanup.policy = compact,delete segment.bytes = 52428800 (~52 mb) min.compaction.lag.ms = 1800000 (30 min) delete.retention.ms = 86400000 (1 day) retention.ms = 259200000 (3 days) Ideally I would want the old records > 3 days to be deleted without producing an explicit delete(null value of a key) of the record. But there can be a case due to continuous compaction, the segments can contain a very old record(eg: > 30 days) and new recent record (eg: 1hr) which will make the segment ineligible for retention delete. Currently I don't see a work around for this. Please suggest. I plan to start a KIP to address this use case. Thanks, Koushik
