re: https://issues.apache.org/jira/browse/KAFKA-7137
My team is investigating what would be involved in code changes to give some finer control over when compaction runs. Detail in the ticket but essentially the current way is that dirty.ratio is hijacked to be set to 0 to give guarantees that a tombstone record will result in previous record being deleted within a set time. This results in a lot of unnecessary compaction, and we would like to add the ability to provide a max delay for compaction to be used in combination with a "healthy" dirty.ratio setting to meet this requirement instead. And to consider an API to trigger compaction (but that is looking difficult because compaction isn't really triggered, it's evaluated by the log cleaner thread). We'd like to get some discussion going around this. Should we draft up a KIP first then kick off a discussion? (if yes, can I get KIP access for: brettr...@gmail.com ) Thanks! -- Brett Rann Senior DevOps Engineer Zendesk International Ltd 395 Collins Street, Melbourne VIC 3000 Australia Mobile: +61 (0) 418 826 017
