Thanks for the KIP Divij! This is a nice improvement to avoid recalculation of size. Customized RLMMs can implement the best possible approach by caching or maintaining the size in an efficient way. But this is not a big concern for the default topic based RLMM as mentioned in the KIP.
~Satish. On Wed, 13 Jul 2022 at 18:48, Divij Vaidya <divijvaidy...@gmail.com> wrote: > Thank you for your review Luke. > > > Reg: is that would the new `RemoteLogSizeBytes` metric be a performance > overhead? Although we move the calculation to a seperate API, we still > can't assume users will implement a light-weight method, right? > > This metric would be logged using the information that is already being > calculated for handling remote retention logic, hence, no additional work > is required to calculate this metric. More specifically, whenever > RemoteLogManager calls getRemoteLogSize API, this metric would be captured. > This API call is made every time RemoteLogManager wants to handle expired > remote log segments (which should be periodic). Does that address your > concern? > > Divij Vaidya > > > > On Tue, Jul 12, 2022 at 11:01 AM Luke Chen <show...@gmail.com> wrote: > > > Hi Divij, > > > > Thanks for the KIP! > > > > I think it makes sense to delegate the responsibility of calculation to > the > > specific RemoteLogMetadataManager implementation. > > But one thing I'm not quite sure, is that would the new > > `RemoteLogSizeBytes` metric be a performance overhead? > > Although we move the calculation to a seperate API, we still can't assume > > users will implement a light-weight method, right? > > > > Thank you. > > Luke > > > > On Fri, Jul 1, 2022 at 5:47 PM Divij Vaidya <divijvaidy...@gmail.com> > > wrote: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-852%3A+Optimize+calculation+of+size+for+log+in+remote+tier > > > > > > > > > Hey folks > > > > > > Please take a look at this KIP which proposes an extension to KIP-405. > > This > > > is my first KIP with Apache Kafka community so any feedback would be > > highly > > > appreciated. > > > > > > Cheers! > > > > > > -- > > > Divij Vaidya > > > Sr. Software Engineer > > > Amazon > > > > > >
