Hi, Luis, Thanks for the KIP. Overall, this seems a useful KIP. A few comments below.
1. I guess both new configurations will be at the topic level? 2. Since the log cleaner now needs to keep both the offset and another long (say timestamp) in the de-dup map, it reduces the number of keys that we can keep in the map and thus may require more rounds of cleaning. This is probably not a big issue, but it will be useful to document this impact in the KIP. 3. With the new cleaning strategy, we want to be a bit careful with removing the last message in a partition (which is possible now). We need to preserve the offset of the last message so that we don't reuse the offset for a different message. One way to simply never remove the last message. Another way (suggested by Jason) is to create an empty message batch. Jun On Sat, Jun 9, 2018 at 12:39 AM, Luís Cabral <luis_cab...@yahoo.com.invalid> wrote: > Hi all, > > Any takers on having a look at this KIP and voting on it? > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 280%3A+Enhanced+log+compaction > > Cheers, > Luis >
