+1 (non-binding). Thanks for the KIP! On Mon, Jan 13, 2020 at 1:58 PM M. Manna <manme...@gmail.com> wrote:
> Hi Mikael, > > Apologies for last minute question, as I just caught up with it. Thanks for > your work on the KIP. > > Just trying to get your thoughts on one thing (I might have misunderstood > it) - currently it's possible (even though I am strongly against it) to > create Kafka topics which are under-replicated; despite all brokers being > online. This the the output of an intentionally under-replicated topic > "dummy" with p=6 and RF=1 (with a 3 node cluster) > > > virtualadmin@kafka-broker-machine-1:/opt/kafka/bin$ ./kafka-topics.sh > --create --topic dummy --partitions 6 --replication-factor 1 > --bootstrap-server localhost:9092 > virtualadmin@kafka-broker-machine-1:/opt/kafka/bin$ ./kafka-topics.sh > --describe --topic dummy --bootstrap-server localhost:9092 > Topic:dummy PartitionCount:6 ReplicationFactor:1 > > Configs:compression.type=gzip,min.insync.replicas=2,cleanup.policy=delete,segment.bytes=10485760,max.message.bytes=10642642,retention.bytes=20971520 > Topic: dummy Partition: 0 Leader: 3 Replicas: 3 > Isr: 3 > Topic: dummy Partition: 1 Leader: 1 Replicas: 1 > Isr: 1 > Topic: dummy Partition: 2 Leader: 2 Replicas: 2 > Isr: 2 > Topic: dummy Partition: 3 Leader: 3 Replicas: 3 > Isr: 3 > Topic: dummy Partition: 4 Leader: 1 Replicas: 1 > Isr: 1 > Topic: dummy Partition: 5 Leader: 2 Replicas: 2 > Isr: 2 > > This is with respect to the following statement on your KIP (i.e. > under-replicated topic creation is also permitted when none is offline): > > *but note that this may already happen (without this KIP) when > > topics/partitions are created while all brokers in a rack are offline > (ie: > > an availability zone is offline). Tracking topics/partitions not > optimally > > spread across all racks can be tackled in a follow up KIP. * > > > > > Did you mean to say that such under-replicated topics (including > human-created ones) will be handled in a separete KIP ? > > Regards, > > > On Mon, 13 Jan 2020 at 10:15, Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > Hi all. > > > > With 2.5.0 approaching, bumping this thread once more as feedback or > > votes would be nice. > > > > Thanks > > > > On Wed, Dec 18, 2019 at 1:59 PM Tom Bentley <tbent...@redhat.com> wrote: > > > > > > +1 non-binding. Thanks! > > > > > > On Wed, Dec 18, 2019 at 1:05 PM Sönke Liebau > > > <soenke.lie...@opencore.com.invalid> wrote: > > > > > > > Hi Mickael, > > > > > > > > thanks for your response! That all makes perfect sense and I cannot > > > > give any actual use cases for where what I asked about would be > useful > > > > :) > > > > It was more the idle thought if this might be low hanging fruit while > > > > changing this anyway to avoid having to circle back later on and > > > > wanted to at least mention it. > > > > > > > > I am totally happy either way! > > > > > > > > Best regards, > > > > Sönke > > > > > > > > On Wed, 18 Dec 2019 at 11:20, Mickael Maison < > mickael.mai...@gmail.com > > > > > > > wrote: > > > > > > > > > > Thanks Sönke for the feedback. > > > > > > > > > > I debated this point quite a bit before deciding to base creation > > > > > around "min.insync.replicas". > > > > > > > > > > For me, the goal of this KIP is to enable administrators to provide > > > > > higher availability. In a 3 node cluster configured for high > > > > > availability (3 replicas, 2 min ISR), by enabling this feature, > > > > > clusters should be fully usable even when 1 broker is down. This > > > > > should cover all "normal" maintenance operations like a rolling > > > > > restart or just the recovery of a broker. > > > > > > > > > > At the moment, when creating a topic/partition, the assumption is > > that > > > > > the resource will be fully functioning. This KIP does not change > this > > > > > assumption. If this is something someone wants, I think it should > be > > > > > handled in a different KIP that targets that use case. By relying > on > > > > > "min.insync.replicas", we don't break any assumptions the user has > > and > > > > > this should be fully transparent from the user point of view. > > > > > > > > > > About "min.insync.replicas", one caveat that is not explicit in the > > > > > KIP is that it's currently possible to create topics with less > > > > > replicas than this settings. For that reason, I think the > > > > > implementation will actually rely on min(replicas, min-isr) instead > > of > > > > > simply min.insync.replicas. I have updated the KIP to explicitly > > > > > mention this point. > > > > > > > > > > I hope that answers your question, let me know. > > > > > Thanks > > > > > > > > > > > > > > > On Mon, Dec 16, 2019 at 4:38 PM Sönke Liebau > > > > > <soenke.lie...@opencore.com.invalid> wrote: > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > that sounds like a useful addition! I can't help but wonder > > whether by > > > > > > leaving in the restriction that "min.insync.replicas" has to be > > > > > > satisfied we'll be back here in a years time because someone has > a > > > > > > scenario where he or she wants to go below that :) > > > > > > I don't have a strong opinion either way to be honest, just a > > random > > > > > > thought when reading the KIP. > > > > > > > > > > > > Best regards, > > > > > > Sönke > > > > > > > > > > > > On Thu, 12 Dec 2019 at 22:44, Ryanne Dolan < > ryannedo...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > +1 non-binding, thx > > > > > > > > > > > > > > On Thu, Dec 12, 2019 at 6:09 AM Mickael Maison < > > > > mickael.mai...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Bumping this thread, I've not seen any votes or feedback. > > > > > > > > > > > > > > > > On Wed, Nov 13, 2019 at 12:17 PM Mickael Maison > > > > > > > > <mickael.mai...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I'd like to start a vote on KIP-409: Allow creating > > > > under-replicated > > > > > > > > > topics and partitions > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-409%3A+Allow+creating+under-replicated+topics+and+partitions > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sönke Liebau > > > > > > Partner > > > > > > Tel. +49 179 7940878 > > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - > > Germany > > > > > > > > > > > > > > > > -- > > > > Sönke Liebau > > > > Partner > > > > Tel. +49 179 7940878 > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > > > > > > > > > > >
