Hi, DJ2: Using "-" sounds good to me.
DJ3: That seems reasonable to me. DJ4: Why not add it right now? I don't want to change the output of the tool too many times. Best, David On Fri, Nov 15, 2024 at 3:23 PM PoAn Yang <yangp...@gmail.com> wrote: > Hi David / Andrew, > > Thanks for review. Thanks Andrew for picking up kafka-share-groups.sh > implementation. > I will handle kafka-consumer-groups.sh. > > DJ3: After discussing with @Chia-Ping Tsai, we think that using new format > is more clear. > > The new format will be like > <topic-name1>:<partition-id1>,<partition-id2>;<topic-name2>:<partition-id1>,<partition-id2> > > Using colon(:) to concat topic name and partition IDs. > Using comma(,) to concat partition IDs within same topic name. > Using semicolon(;) to concat topic strings. > > AS3: Fix it with kafka-share-groups.sh. Thanks. > > If there is no further suggestion, I will start a vote thread next Monday. > > Thanks, > PoAn > > > On Nov 14, 2024, at 9:05 PM, Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > > Hi PoAn, > > DJ2: I was just going to comment that "-" would be a more appropriate > missing value, but > > you got there first. > > > > AS3: The examples for kafka-share-groups.sh include > kafka-consumer-groups.sh in the > > command line. > > > > If this is accepted in time, I'm happy to pick up the implementation of > the share groups > > part of this if it helps. > > > > Thanks, > > Andrew > > > > ________________________________________ > > From: Frank Yang <yangp...@gmail.com> > > Sent: 14 November 2024 10:48 > > To: dev@kafka.apache.org <dev@kafka.apache.org> > > Subject: Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command > line tool to support new consumer group > > > > Hi David, > > > > Thanks for the review and suggestion! I would like to get this in AK 4.0 > as well. I will do my best. > > > > DJ1: Update KIP to put GROUP-EPOCH and TARGET-ASSIGNMENT-EPOCH before > #MEMBERS. > > > > DJ2: I prefer to follow current missing column value “-“. (reference < > https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L92 > >) > > > > DJ3: Update KIP to use CURRENT-EPOCH CURRENT-ASSIGNMENT TARGET-EPOCH > TARGET-ASSIGNMENT. > > Remove GROUP-EPOCH. > > > > For assignment value, it follows current output (reference < > https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L413-L418>). > I think the form of `topicid-partitionid` is more clear. > > If we would like to use this form, I will update both output in > kafka-consumer-groups.sh and kafka-share-groups.sh. > > > > DJ4: It looks like DescribeGroupsResponseData only has protocol type at > group level. > > Both DescribeGroupsResponseData and ConsumerGroupDescribeResponseData > don’t have protocol at member level. > > Could we use a followup to add it? > > > > DJ5: Update KIP to put LEADER-EPOCH before CURRENT-OFFSET. > > > > Thanks, > > PoAn > > > >> On Nov 14, 2024, at 3:43 PM, David Jacot <dja...@confluent.io.INVALID> > wrote: > >> > >> Hi PoAn, > >> > >> Thanks for the KIP! I have a few minor comments/suggestions: > >> > >> DJ1: In the output of `--describe --verbose`, I would suggest putting > >> `GROUP-EPOCH` and `TARGET-ASSIGNMENT-EPOCH` before `#MEMBERS`. > >> DJ2: Continuing on the above, I assume that we will print out N/A for > the > >> fields not supported by classic groups. Is this correct? > >> DJ3: In the output of `--describe --members --verbose`, I wonder if we > >> should use the following order and terms: CURRENT-EPOCH > CURRENT-ASSIGNMENT > >> TARGET-EPOCH TARGET-ASSIGNMENT . I would remove the GROUP-EPOCH because > it > >> is already in the group description. The value `(0)` used for the > >> assignment is incorrect. Here I suppose that we will print out the list > of > >> partitions in the form of `topicid-partitionid`. > >> DJ4: Continuing on the above, I wonder if we should also add the > protocol > >> used `classic` or `consumer`. For context, it is possible to have > `classic` > >> members and `consumer` members in a `consumer` group during an online > >> upgrade from the classic protocol to the consumer protocol. Having this > >> information may be handy for administrators. What do you think? > >> DJ5: In the output of `--describe --offsets --verbose`, I would suggest > >> putting `LEADER-EPOCH` closer to `CURRENT-OFFSET`. > >> > >> It would be great if we could get this one in AK 4.0 as it changes the > >> output of the command. > >> > >> Thanks, > >> DJ > >> > >> On Fri, Nov 1, 2024 at 7:40 AM Frank Yang <yangp...@gmail.com> wrote: > >> > >>> Hi Sean / Andrew / Lianet, > >>> > >>> Thanks for all your review and suggestions. > >>> > >>> AS1, LM1, LM4: Change to add KIP-848 information when users give > —verbose > >>> option. > >>> —describe —verbose: shows group epoch and target assignment epoch. > >>> —describe —members —verbose: shows above information, member epoch, and > >>> target assignment. > >>> > >>> AS2: Change to use MEMBER-EPOCH to align with KIP-848 definition. > >>> > >>> LM2: For classic group, it doesn’t have epoch, so I use Optional > fields in > >>> ConsumerGroupDescription. > >>> For share group, it relies on KIP-848. It must have epoch, so I use int > >>> fields in ShareGroupDescription. > >>> > >>> LM3: Remove —state change. We can get group level information by > —describe > >>> —verbose. > >>> > >>> SQ1: Add LEADER-EPOCH when users give —describe —offsets —verbose. > >>> > >>> Thanks. > >>> PoAn > >>> > >>>> On Nov 1, 2024, at 5:08 AM, Lianet M. <liane...@gmail.com> wrote: > >>>> > >>>> Hello Frank, thanks for the KIP! A few comments: > >>>> > >>>> LM1. I strongly agree with Andrew's suggestion of moving this into a > >>>> --verbose option. I expect someone would only attempt to make sense of > >>> the > >>>> epochs while debugging an issue, not while trying to view the group or > >>>> member's info. (Also member-epoch makes more sense to me, even more > if we > >>>> end up agreeing in a --verbose). Related to both issues, my take is > that > >>>> whoever is close to the protocol will expect member-epoch, whoever is > not > >>>> will probably won't even need to see the epochs at all. > >>>> > >>>> LM2. Why are the epochs defined as Optional (don't we expect to always > >>> have > >>>> them? with 0 initially, defined on the broker side for the group ones, > >>> and > >>>> client side for the member) > >>>> > >>>> LM3. Why is the KIP including the –state option in the proposed > changes? > >>> ( > >>>> > >>> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=327977411#KIP1099:Extendkafkaconsumergroupscommandlinetooltosupportnewconsumergroup---state > >>>> ). > >>>> I get that the output would change in that example, but it’s not > because > >>> of > >>>> any change to the –state option itself. It's because of the change > >>> proposed > >>>> to the –described (required with the --state), and the changes to > >>>> --describe are already explained above (seems confusing, got me > looking > >>> for > >>>> a change to the state filter which seemed unrelated). > >>>> > >>>> LM4. Group epoch and target assignment epoch are conceptually at the > >>> group > >>>> level. vs member epoch that lands at a member level. So wonder if we > >>> should > >>>> show them accordingly (ex. using the --verbose option) > >>>> –describe –verbose => shows group epoch and target assignment epoch > >>>> –describe –members –verbose => shows member epoch (along > >>>> with group epoch and target assignment epoch) > >>>> > >>>> Thanks! > >>>> > >>>> Lianet > >>>> > >>>> On Thu, Oct 31, 2024 at 7:30 AM Andrew Schofield < > >>>> andrew_schofield_j...@outlook.com> wrote: > >>>> > >>>>> Hi PoAn, > >>>>> Thanks for the KIP. I have a few comments. > >>>>> > >>>>> AS1: It seems to me that these new additions are most useful to > people > >>>>> trying to understand > >>>>> the progress of rebalancing in quite some detail. The information is > >>>>> really not understandable > >>>>> for most users who do not have deep knowledge of KIP-848/932. > >>>>> > >>>>> As a result, I suggest for kafka-share-groups.sh that you add a > >>> --members > >>>>> --verbose option > >>>>> and only include the new information in the output for that option, > >>> rather > >>>>> than changing the > >>>>> non-verbose --members output. > >>>>> > >>>>> I also make a similar suggestion for kafka-consumer-groups.sh > --members > >>>>> and only add the > >>>>> new information for the --verbose output. > >>>>> > >>>>> AS2: I strongly suggest that you use MEMBER-EPOCH instead of > >>>>> CONSUMER-EPOCH. > >>>>> I think it's more confusing not following the terminology of the > KIPs. > >>> For > >>>>> one thing, > >>>>> we already have "member" in the admin client such as > MemberDescription. > >>>>> It's not a > >>>>> new term. > >>>>> > >>>>> > >>>>> Thanks, > >>>>> Andrew > >>>>> ________________________________________ > >>>>> From: PoAn Yang <pay...@apache.org> > >>>>> Sent: 25 October 2024 13:55 > >>>>> To: dev@kafka.apache.org <dev@kafka.apache.org> > >>>>> Subject: Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command > >>> line > >>>>> tool to support new consumer group > >>>>> > >>>>> Hi Lucas, > >>>>> > >>>>> Thanks for the review! > >>>>> > >>>>> 1) Yes, I add related change for kafka-share-groups.sh to the KIP. > Could > >>>>> you take a look? Thanks for the suggestion. > >>>>> > >>>>> 2) We use CONSUMER-ID as member ID. If we use MEMBER-EPOCH here, > users > >>> may > >>>>> confuse what is different between CONSUMER and MEMBER. > >>>>> > >>>>> Thanks, > >>>>> PoAn > >>>>> > >>>>> On 2024/10/23 13:28:17 Lucas Brutschy wrote: > >>>>>> Hi Frank, > >>>>>> > >>>>>> thanks for the KIP! > >>>>>> > >>>>>> 1) For consistency, should we do the same for > >>>>>> kafka-share-groups.sh, ShareGroupDescription, etc. ? Even if we do > not > >>>>>> implement it right now if the share group implementation may still > be > >>>>>> incomplete, it may make sense to include it in the KIP. > >>>>>> > >>>>>> 2) Why call it CONSUMER-EPOCH, not MEMBER-EPOCH? That would seem > more > >>>>>> consistent. > >>>>>> > >>>>>> Cheers, > >>>>>> Lucas > >>>>>> > >>>>>> On Wed, Oct 23, 2024 at 2:41 PM Frank Yang <yangp...@gmail.com> > wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I would like to kick off the discussion of KIP-1099. This KIP > enhances > >>>>> the > >>>>>>> kafka-consumer-groups tools to include state which is introduced by > >>>>> KIP-848. > >>>>>>> > >>>>>>> KIP-1099: Extend kafka-consumer-groups command line tool to support > >>> new > >>>>>>> consumer group - Apache Kafka - Apache Software Foundation > >>>>>>> < > >>>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>> > >>>>>>> cwiki.apache.org > >>>>>>> < > >>>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>> > >>>>>>> [image: favicon.ico] > >>>>>>> < > >>>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>> > >>>>>>> < > >>>>> > >>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>> > >>>>>>> > >>>>>>> Thank you, > >>>>>>> PoAn > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > >
