Hi Colin,
Thanks for considering the idea and sharing your feedback.
The improvements I proposed can be achieved, to some extend, using the
AdminClient API and the Consumer Group CLI tool. But they won't fully
support the proposal.
For example,
Regular expressions are not supported on the groups
Topic / Client filtering is not supported across all groups
So the reason for proposing the idea was to see if other Kafka users are
also interested in some of these features so we can remove the burden of
them writing custom code around existing consumer group features, and make
those features built into Kafka Consumer Group Command and AdminClient
API.
Thanks again!
--Vahid
From: Colin McCabe <cmcc...@apache.org>
To: us...@kafka.apache.org
Date: 08/03/2018 04:16 PM
Subject: Re: ConsumerGroupCommand tool improvement?
Hi Vahid,
Interesting idea.
It seems like if you're using the AdminClient APIs programmatically, you
can just do the filtering yourself in a more flexible way than what we
could provide.
On the other hand, if you're using the ./bin/consumer-groups.sh
command-line tool, why not use grep or a similar tool to filter the
output? Maybe there is some additional functionality in supporting
regexes in the command-line tool, but it also seems like it might be kind
of complex as well. Do you have some examples where having regex support
int the tool would be much easier than the traditional way of piping the
output to grep, awk, and sed?
best,
Colin
On Thu, Aug 2, 2018, at 14:23, Vahid S Hashemian wrote:
> Hi all,
>
> A requirement has been raised by a colleague and I wanted to see if
there
> is any interest in the community in adding the functionality to Apache
> Kafka.
>
> ConsumerGroupCommand tool in describe ('--describe' or '--describe
> --offsets') mode currently lists all topics the group has consumed from
> and all consumers with assigned partitions for a single group.
> The idea is to allow filtering of topics, consumers (client ids), and
even
> groups using regular expressions. This will allow the tool to handle use
> cases such as
> What's the status of a particular consumer (or consumers) in all the
> groups they are consuming from? (for example to check if they are
lagging
> behind in all groups)
> What consumer groups are consuming from a topic (or topics) and what's
the
> lag for each group?
> Limit the existing result to the topics/consumers of interest (for
groups
> with several topics/consumers)
> ...
>
> This would potentially lead to enhancing the AdminClient API as well.
>
> If the community also sees a value in this, I could start drafting a
KIP.
>
> Thanks for your feedback.
> --Vahid
>
