Oops sorry looks like graph didn’t work? Raw result for 1GB/s in, and 1GB/s out ($/month for inter AZ costs on AWS with AZ=RF=3) are:
default replication no follower fetching $/month 175334.4 default replication follower fetching $/month 140267.52 simple diskless KIP-1176 without follower fetching $/month 70133.76 simple diskless KIP-1176 with follower fetching $/month 35066.88 complex diskless KIP-1150 $/month 0 Paul From: Brebner, Paul <paul.breb...@netapp.com.INVALID> Date: Friday, 8 August 2025 at 10:04 pm To: us...@kafka.apache.org <us...@kafka.apache.org>, dev <dev@kafka.apache.org> Subject: Re: [DISCUSS] The Path Forward for Saving Cross-AZ Costs KIPs EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Initial POC cost modelling (just for inter AZ costs, not overall costs) results are in: [image.png] From: Brebner, Paul <paul.breb...@netapp.com.INVALID> Date: Friday, 8 August 2025 at 12:37 pm To: us...@kafka.apache.org <us...@kafka.apache.org>, dev <dev@kafka.apache.org> Subject: Re: [DISCUSS] The Path Forward for Saving Cross-AZ Costs KIPs EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Thanks Luke, I’m taking a closer look at these now … Paul From: Luke Chen <show...@gmail.com> Date: Tuesday, 5 August 2025 at 6:32 pm To: dev <dev@kafka.apache.org>, Kafka Users <us...@kafka.apache.org> Subject: [DISCUSS] The Path Forward for Saving Cross-AZ Costs KIPs EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hi all, The Kafka community is currently seeing an unprecedented situation with three KIPs (KIP-1150, IP-1176, KIP-1183) simultaneously addressing the same challenge of high replication costs when running Kafka across multiple cloud availability zones. Each KIP offers a different solution to this issue. While diversity of innovative ideas is a key strength of open-source projects, it creates a burden for reviewers and users who must compare and comment on multiple proposals simultaneously. Furthermore, discussion around the three KIPs has stalled for over two months now. This could be due to the authors being hesitant to proceed due to the existence of alternative, potentially conflicting, solutions. Addressing replication cost is a key concern of Kafka’s userbase and we should try to move the conversation forward if we can. From what I understand, these three KIPs are not mutually exclusive. But adopting all three KIPs in the community might not be what we expect. Thus, I would like to *start a discussion on how we could move the conversation forward*. To save time for the KIP readers/reviewers, I have created this document <https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/The*Path*Forward*for*Saving*Cross-AZ*Replication*Costs*KIPs__;KysrKysrKys!!Nhn8V6BzJA!QwziXdTS-KWgnwDY2e2YMZ1ECw0bksHP5SHngXwODpyooLqd-Qe1ufCKI5gL49b4MAdRfujIvmmSlTD6G8DP$ >[1] to help summarize each of the KIPs and describe their current status. *Hope to get some suggestions/feedback from the community*. [1] https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/The*Path*Forward*for*Saving*Cross-AZ*Replication*Costs*KIPs__;KysrKysrKys!!Nhn8V6BzJA!QwziXdTS-KWgnwDY2e2YMZ1ECw0bksHP5SHngXwODpyooLqd-Qe1ufCKI5gL49b4MAdRfujIvmmSlTD6G8DP$ KIP-1150: https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1150*3A*Diskless*Topics__;JSsr!!Nhn8V6BzJA!QwziXdTS-KWgnwDY2e2YMZ1ECw0bksHP5SHngXwODpyooLqd-Qe1ufCKI5gL49b4MAdRfujIvmmSlcbarKrW$ KIP-1176: https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1176*3A*Tiered*Storage*for*Active*Log*Segment__;JSsrKysrKw!!Nhn8V6BzJA!QwziXdTS-KWgnwDY2e2YMZ1ECw0bksHP5SHngXwODpyooLqd-Qe1ufCKI5gL49b4MAdRfujIvmmSlXgJtRjr$ KIP-1183: https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1183*3A*Unified*Shared*Storage__;JSsrKw!!Nhn8V6BzJA!QwziXdTS-KWgnwDY2e2YMZ1ECw0bksHP5SHngXwODpyooLqd-Qe1ufCKI5gL49b4MAdRfujIvmmSlRhshGC8$ Thank you. Luke
