Ping on this. We're prepared to do the work i'm just not sure of the right way to start. If nobody wants to discuss it now should we just write up a KIP?
On Thu, Jul 19, 2018 at 10:57 AM Brett Rann <br...@zendesk.com> wrote: > re: https://issues.apache.org/jira/browse/KAFKA-7137 > > My team is investigating what would be involved in code changes to give > some finer control over when compaction runs. Detail in the ticket but > essentially the current way is that dirty.ratio is hijacked to be set to 0 > to give guarantees that a tombstone record will result in previous record > being deleted within a set time. > > This results in a lot of unnecessary compaction, and we would like to add > the ability to provide a max delay for compaction to be used in combination > with a "healthy" dirty.ratio setting to meet this requirement instead. > > And to consider an API to trigger compaction (but that is looking > difficult because compaction isn't really triggered, it's evaluated by the > log cleaner thread). > > We'd like to get some discussion going around this. Should we draft up a > KIP first then kick off a discussion? (if yes, can I get KIP access for: > brettr...@gmail.com ) > > Thanks! > > -- > > Brett Rann > > Senior DevOps Engineer > > > Zendesk International Ltd > > 395 Collins Street, Melbourne VIC 3000 Australia > > Mobile: +61 (0) 418 826 017 > -- Brett Rann Senior DevOps Engineer Zendesk International Ltd 395 Collins Street, Melbourne VIC 3000 Australia Mobile: +61 (0) 418 826 017
