During the vote it was suggested the name of the property be changed from:
log.cleaner.compaction.lag.ms
to
log.cleaner.compaction.delay.ms
Note that this feature concerns controlling the portion of the head of the
log (new messages go on the head) that will be left un-compacted (i.e. in
full detail) in the event that the log undergoes a compaction. This
property is not saying *when* to do a compaction rather it is how far a
compaction may go into the head. A compaction can still happen to the log
so we are not "delaying" the compaction itself, rather we are preventing
the compaction from proceeding all the way to the head of the log.
One other consideration
The original name ("lag") was proposed when non-temporal constraints were
being considered (size, # of messages). Those were deferred in the KIP
process. However, the new name ("delay") is (to my ear) purely temporal. In
the event use cases for other forms of compaction point configuration do
arise naming them consistently may be trickier.
To James Cheng's concern:
I just thought of something: what happens to the value of "
> log.cleaner.delete.retention.ms"?
> Does it still have the same meaning as before? Does the timer start when
> log compaction happens
> (as it currently does), so in reality, tombstones will only be removed
> from the log some time
> after (log.cleaner.min.compaction.lag.ms + log.cleaner.delete.retention.ms
> )?
Nothing about tombstones handling changes. The tombstone timer starts when
the tombstone is created (i.e. during compaction). As long as one
interprets "log.cleaner.delete.retention.ms" as the minimum time-to-live of
the tombstones then the meaning of that property doesn't change. The new
feature only indirectly makes the tombstone removal later by virtue of
delaying their creation.
