Hi Rajini, Thanks for the KIP. I have a few questions/comments:
01. If I understood correctly, the plan is to add new assignors which are rack aware. Is this right? I wonder if it is a judicious choice here. The main drawback is that clients must be configured correctly in order to get the benefits. From a user experience perspective, it would be much better if we would only require our users to set client.rack. However, I understand the argument of keeping the existing assignors as-is in order to limit the risk but it also means that we will have to maintain multiple assignors with a somewhat similar core logic (within a rack). What do you think? 02. If we proceed with new rack-aware assignors, we should mention their fully qualified names in the KIP as they will become part of our public interfaces. 03. KIP-792 has already introduced version 2 of the subscription. The KIP is accepted but the PR is not merged yet. This KIP should use version 3. Best, David On Thu, Nov 3, 2022 at 5:58 PM Viktor Somogyi-Vass <viktor.somo...@cloudera.com.invalid> wrote: > > Hi Rajini, > > If I understand correctly, the client.rack config would stay supported > after KIP-848 but does it expand the scope of that KIP too with this > config? I mean that currently you propose ConsumerProtocolSubscription to > be used but this protocol won't be available and we need to transfer the > config to the coordinator via other means. Should this be added to that KIP? > > Thanks, > Viktor > > On Wed, Nov 2, 2022 at 9:50 PM Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > > > Hi Jun, > > > > Thank you for the review. Yes, we should add rack id to Subscription, had > > missed that part. Updated the KIP, thank you for pointing that out! > > > > Regards, > > > > Rajini > > > > On Wed, Nov 2, 2022 at 7:06 PM Jun Rao <j...@confluent.io.invalid> wrote: > > > > > Hi, Rajini, > > > > > > Thanks for the KIP. Just one comment. > > > > > > Should we add rackId to GroupSubscription.Subscription for each member? > > > > > > Thanks, > > > > > > Jun > > > > > > On Wed, Nov 2, 2022 at 4:57 AM Rajini Sivaram <rajinisiva...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I have submitted KIP-881 to implement rack-aware partition assignment > > for > > > > consumers: > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-881%3A+Rack-aware+Partition+Assignment+for+Kafka+Consumers > > > > . > > > > It adds rack id to the consumer group protocol to propagate rack > > > > information so that rack-aware assignors can be added to benefit from > > > > locality. > > > > > > > > Feedback and suggestions are welcome! > > > > > > > > Thank you, > > > > > > > > Rajini > > > > > > > > >
