Hey Gwen/Vahid, I think that use case makes sense, but isn't it a little odd to go through the consumer group tool? I would expect to find something like that from the kafka-topics tool instead. I don't feel too strongly about it, but I hate to complicate the tool by adding the need to query topic configs. If we don't have a meaningful number to report for compacted topics anyway, then it feels like only a half solution. I'd probably suggest leaving this out or just reporting the absolute difference even if a topic is compacted.
-Jason On Thu, Jun 28, 2018 at 1:05 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi James, > > > > Thanks for the feedback. I updated the KIP and added some of the benefits > > of this improvement (including some that you mentioned). > > > > Regards. > > --Vahid > > > > > > > > From: James Cheng <wushuja...@gmail.com> > > To: dev@kafka.apache.org > > Date: 06/27/2018 09:13 PM > > Subject: Re: [DISCUSS] KIP-325: Extend Consumer Group Command to > > Show Beginning Offsets > > > > > > > > The “Motivation” section of the KIP says that the starting offset will be > > useful but doesn’t say why. Can you add a use-case or two to describe how > > it will be useful? > > > > In our case (see > > https://github.com/wushujames/kafka-utilities/blob/master/Co > nsumerGroupLag/README.md > > ), we found the starting offset useful so that we could calculate > > partition size so that we could identify empty partitions (partitions > > where all the data had expired). In particular, we needed that info so > > that we could calculate “lag”. Consider that case where we are asked to > > mirror an abandoned topic where startOffset==endOffset==1000000. We would > > have no committed offset on it, and the topic has no data in it, so we > > won’t soon get any committed offset, and so “lag” is kind of undefined. We > > used the additional startOffset to allow us to detect this case. > > > > -James > > > > Sent from my iPhone > > > > > On Jun 26, 2018, at 11:23 AM, Vahid S Hashemian > > <vahidhashem...@us.ibm.com> wrote: > > > > > > Hi everyone, > > > > > > I have created a trivial KIP to improve the offset reporting of the > > > consumer group command: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-325% > 3A+Extend+Consumer+Group+Command+to+Show+Beginning+Offsets > > > > > Looking forward to your feedback! > > > > > > Thanks. > > > --Vahid > > > > > > > > > > > > > > > >
