Hey there, Thanks for sharing this KIP. I took a brief look and had a similar thought as David about this with wondering how this compares to using tiered storage. I was also thinking about fetching from any replica (KIP-392) related to this as well.
I think overall, Chunk could be useful, but it is a tradeoff to increase metadata complexity. Justine On Wed, Dec 11, 2024 at 11:11 AM De Gao <d...@live.co.uk> wrote: > Hi David: > > Thanks for the helping info. Appreciated. > I believe the chunk is still needed in spite of tiered storage as this > makes Kafka more 'complete'. Let me extend the motivation section. > I wasn't aware we are busy on Kafka 4.0. If that's the case I will see if > can contribute on that while waiting for reviews. > > Regards > > De Gao > > On 11 December 2024 08:51:43 GMT, David Jacot <dja...@confluent.io.INVALID> > wrote: > >Hi, > > > >Thanks for the KIP. The community is pretty busy with the Apache Kafka 4.0 > >release so I suppose that no one really had the time to engage in > reviewing > >the KIP yet. Sorry for this! > > > >I just read the motivation section. I think that it is an interesting > idea. > >However, I wonder if this is still needed now that we have tier storage in > >place. One of the big selling points of tier storage was that clusters > >don't have to replicate tiered data anymore. Could you perhaps extend the > >motivation of the KIP to include tier storage in the reflexion? > > > >Best, > >David > > > >On Tue, Dec 10, 2024 at 10:46 PM De Gao <d...@live.co.uk> wrote: > > > >> Hi All: > >> > >> There were no discussion in the past week. Just want to double check if > I > >> missed anything? > >> What should be the expectations on KIP discussion? > >> > >> Thank you! > >> > >> De Gao > >> > >> On 1 December 2024 19:36:37 GMT, De Gao <d...@live.co.uk> wrote: > >> >Hi All: > >> > > >> >I would like to start the discussion of KIP-1114 Introducing Chunk in > >> Partition. > >> > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1114%3A+Introducing+Chunk+in+Partition > >> >This KIP is complicated so I expect discussion will take longer time. > >> > > >> >Thank you in advance. > >> > > >> >De Gao > >> >
