Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-27 Thread Josep Prat
min time, deletion is lazy – can occur at some > > > (distant?) > > > > time in the future (and is async I think) – this was particularly > > > > noticeable for tiered storage (only time I’ve really understood how > > Kafka > > > > segments work

Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-27 Thread אורי אהרוני
ered storage (only time I’ve really understood how > Kafka > > > segments work and looked closely), Paul > > > > > > From: Matthias J. Sax > > > Date: Tuesday, 25 February 2025 at 1:16 pm > > > To: users@kafka.apache.org > > > Subject:

Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-27 Thread Josep Prat
> time in the future (and is async I think) – this was particularly > > noticeable for tiered storage (only time I’ve really understood how Kafka > > segments work and looked closely), Paul > > > > From: Matthias J. Sax > > Date: Tuesday, 25 February 2025 at

Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-27 Thread אורי אהרוני
iceable for tiered storage (only time I’ve really understood how Kafka > segments work and looked closely), Paul > > From: Matthias J. Sax > Date: Tuesday, 25 February 2025 at 1:16 pm > To: users@kafka.apache.org > Subject: Re: Documentation and meaning of configuration 'ret

Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-25 Thread Brebner, Paul
tiered storage (only time I’ve really understood how Kafka segments work and looked closely), Paul From: Matthias J. Sax Date: Tuesday, 25 February 2025 at 1:16 pm To: users@kafka.apache.org Subject: Re: Documentation and meaning of configuration 'retention.bytes' EXTERNAL EMAIL - U

Re: Documentation and meaning of configuration 'retention.bytes'

2025-02-24 Thread Matthias J. Sax
I think you are right. Technically, it's a "minimum" not a "maximum". The cleanup happens async by the background log-cleaner thread. Segments which go beyond the "retention.bytes" config can be removed. I think it's just a difference between "technically correct" (ie, engineering / nerd lang

Documentation and meaning of configuration 'retention.bytes'

2025-02-23 Thread אורי אהרוני
Hi, I encountered a misunderstanding and I would like you to explain it to me or if possible change the documentation. The Kafka docs describes 'retention.bytes' configuration as: This configuration controls the maximum size a partition (which consists of log segments) can grow to before we will d