Someone will need to correct me if I’m wrong, but my understanding is that a topic log on disk is divided into segments. Compaction will occur when a segment “rolls off” - so when a new active segment is created and the previous segment becomes inactive.
Segments can be bounded by size and time in topic and broker configuration to get the effect that you want. > On Jan 19, 2018, at 2:10 PM, Rahul Bhattacharjee <rahul.rec....@gmail.com> > wrote: > > Let's say we have a compacted topic (log.cleanup.policy=compact) where lot > of updates happen for relatively small set of keys. > My question is when does the compaction happen. > > In memtable , when a new update comes for an already existing key in > memtable , the value is simple replaced. > or, > all the updates are associated with a offset , later the memtable is > spilled to disk and the deletion happens during compaction phase. > > thanks, > Rahul
