at 2:00 AM Henry Haiying Cai
wrote:
> Xinyu,
> Thanks for your time reading the KIP and detailed comments. We are
> honored to have technical leaders from AutoMQ to look at our work.
> Please see my answers below inline.
>
> On Tuesday, May 6, 2025 at 08:37:22 PM
Dear Kafka Community,
I am proposing a new KIP to introduce a unified shared storage
solution for Kafka, aiming
to enhance its scalability and flexibility. This KIP is inspired by
the ongoing discussions
around KIP-1150 and KIP-1176, which explore leveraging object storage
to achieve cost and
elas
Hi Henry and Tom,
I've read the entire KIP-1176, and I think it's a smart move to advance
tiered storage.
If I understand correctly, KIP-1176 aims to eliminate cross-AZ traffic in
tier 1 storage by replicating data to followers through the S3EOZ bucket.
After that, followers only need to replicat
vation about storing the metadata in KRaft, instead of
> another internal topic or creating another interface or something else?
> (d) The detail of messages, i.e. GET_KVS, PUT_KVS, ... etc.
>
>
> Thank you.
> Luke
>
> On Tue, May 13, 2025 at 8:22 PM Xinyu Zhou wrote:
>
>
ata may need to be loaded on the new node?
>
> SD-5: In this approach, much of the core functionality resides within
> the Stream implementation. It might be worth considering a design
> where the building blocks handle most of the complexity, allowing the
> plugin to remain as lightweight as p
ds to support both local disk and shared storage long-term.
2. Which path should we take? The leaderless architecture of 1150 or the
approach mentioned in 1183.
I will update the KIP with our discussion soon. Thanks again for your time!
Best,
Xinyu
On Fri, May 16, 2025 at 7:33 AM Colin McCabe wrote:
Hi Luke,
Thank you for creating this dedicated thread; we definitely need a space to
discuss future steps for these topics. I apologize for my delay on KIP-1183
and will provide more details in the coming weeks.
I agree with Stanislav that we should first focus on the community's
direction. Speci
