Hi Murali, Overall LGTM. It would be my honor to give +1 on the vote thread :)
One nit: what happens if the follower's segments don't match the leader's? For example, offsets 0-100 are in two segments on the leader but in a single segment on the follower. Which upload time should the follower pick? Best, Chia-Ping On Wed, Nov 18, 2026 03:14 AM, Muralidhar Basani via dev < [email protected]> wrote: > Hi all, > > I have updated the KIP based on the feedback so far: > > - Compatibility: added a "Disk sizing" section. For backfill data, local > disk grows by about local.retention.ms * write rate per partition replica, > with local.retention.bytes as the way to cap it. > - Proposed Changes: clarified where the upload time comes from, and that > local time-based deletion pauses while the remote log metadata cache is > initializing, with no fallback to record timestamps. > - Rejected Alternatives: added min(record timestamp, upload time). > > Please take a look and let me know if there are further comments. > > Thanks, > Murali > > On Fri, Nov 13, 2026 at 10:29 PM Muralidhar Basani < > [email protected]> wrote: > > > Hi Viktor, > > > > Thanks for the review, and nice point on disk sizing. > > > > Unless there are objections, I shall keep the uploadTime as base, add the > > sizing note to compatibility, and move min(..) to rejected alternatives. > > > > I will update KIP. > > > > Chia, pls let me know if this addresses the concern you raised on Case C. > > > > Thanks, > > Murali > > > > On Thu, Nov 12, 2026 at 11:32 AM Viktor Somogyi-Vass < > > [email protected]> wrote: > > > >> Hi Murali, > >> > >> I also agree with Chia-Ping, taking uploadTime as the new base for > >> local.retention.ms would generally increase local disk usage which > might > >> be surprising and might have disk sizing consequences as well. So the > >> extra local disk required would be local.retention.ms * upload rate, I > >> think we can add this to the KIP. I don't think that changing behavior > is a > >> blocker if it results in a more predictable behavior. > >> I was also considering min(maxTimestamp, uploadTime), but I think that > >> overall it would complicate the config itself, which I would not take > >> compared to uploadTime only as that would create a common, broker-bound > and > >> predictable interpretation for local.retention.ms. > >> > >> Best, > >> Viktor > >> > >> On Thu, Nov 5, 2026 at 10:03 PM Muralidhar Basani via dev < > >> [email protected]> wrote: > >> > >>> Thanks Chia for the review. > >>> > >>> 1. Case C: Yes, this is a behavior change. It only affects topics that > >>> set > >>> local.retention.ms lower than the age of the backfilled data. > >>> With the default (-2 for local.retention.ms, means use the same value > as > >>> retention.ms), backfilled data isn't deleted right after upload today > >>> either. > >>> For example, with retention.ms = 7 days and local.retention.ms left at > >>> its > >>> default of -2, local retention is also 7 days. Data stays on local disk > >>> as > >>> long as it's kept at all. > >>> > >>> I see 2 options. > >>> > >>> a. Keep upload-time semantics and document the change in the upgrade > >>> notes. > >>> Operators who want to bound local disk can set local.retention.bytes: > it > >>> is > >>> per partition, ignores timestamps, and deletes the oldest uploaded > >>> segments > >>> once the limit is exceeded. For a one-off backfill, they can also lower > >>> local.retention.ms on the topic temporarily. > >>> > >>> Ex for bytes : As the limit applies to each partition separately, with > >>> 5gb > >>> per partition, and with 10 partitions, broker can hold upto 50gb > locally. > >>> As it only looks at size, it will ignore the timestamps then. > >>> > >>> b. What do you think of an alternative which is to measure from > >>> min(record > >>> timestamp, upload time). That keeps today's backfill behavior and still > >>> fixes future timestamps, but local.retention.ms would no longer mean > the > >>> same thing for all data. > >>> > >>> Which one do you think fits operators better? > >>> > >>> 2. Metadata cache initializing: No, we won't fall back. RLM tasks skip > >>> their run until RemoteLogMetadataManager#isReady is true, so > >>> highestOffsetInRemoteStorage stays -1 and the upload-before-delete > guard > >>> blocks time-based local deletion. > >>> Basically deletion is paused until cache is ready. So the fallback > isn't > >>> triggered. > >>> > >>> Pls let me know your thoughts. > >>> > >>> Thanks, > >>> Murali > >>> > >>> On Thu, Nov 5, 2026 at 7:06 PM Chia-Ping Tsai via dev < > >>> [email protected]> > >>> wrote: > >>> > >>> > Hi Murali, > >>> > > >>> > I'm worried about Case C: "the local copy is deleted around 11:05, 1 > >>> hour > >>> > after the upload." > >>> > > >>> > Would this surprise operators who assume backfilled data is removed > >>> right > >>> > after upload? I guess the suitable approach for them is to use > >>> > local.retention.bytes? > >>> > > >>> > Also, what happens if the RemoteLogMetadataCache is still > initializing > >>> > when the broker has just started? Will we always take the fallback? > >>> > > >>> > Best, > >>> > Chia-Ping > >>> > > >>> > > >>> > > >>> > On 2026-10-23 18:30 UTC Muralidhar Basani via dev wrote: > >>> > > Hi all, > >>> > > > >>> > > I would like to start a discussion about tiered storage topics and > >>> how > >>> > > local segments are deleted by time, once they are uploaded to > remote > >>> > > storage. > >>> > > > >>> > > > >>> > > >>> https://cwiki.apache.org/confluence/spaces/KAFKA/pages/45197 > 9130/KIP-1385+Measure+local.retention.ms+from+remote+ > upload+time+instead+of+record+timestamps > >>> > > > >>> > > Looking forward to your thoughts on this approach. > >>> > > > >>> > > Thanks, > >>> > > Murali > >>> > > > >>> > > >>> > >> >
