Thanks for the feedback! @Onur, @Gwen:
Agree. Actually at the first draft I considered to have it inside ´kafka-consumer-groups.sh´, but I decide to propose it as a standalone tool to describe it clearly and focus it on reset functionality. But now that you mentioned, it does make sense to have it in ´kafka-consumer-groups.sh´. How would be a consistent way to introduce it? Maybe something like this: ´kafka-consumer-groups.sh --reset-offset --generate --group cg1 --topics t1 --reset-from 2017-01-01T00:00:00.000 --output plan.json´ ´kafka-consumer-groups.sh --reset-offset --verify --reset-json-file plan.json´ ´kafka-consumer-groups.sh --reset-offset --execute --reset-json-file plan.json´ ´kafka-consumer-groups.sh --reset-offset --generate-and-execute --group cg1 --topics t1 --reset-from 2017-01-01T00:00:00.000´ @Gwen: > It looks exactly like the replica assignment tool It was influenced by ;-) I use the generate-verify-execute process here to make sure user will be aware of the result of this operation. At the beginning we considered only add a couple of options to Consumer Group Command: --rewind-to-timestamp and --rewind-to-period @Onur: > You can actually get away with overriding while members of the group are live with method 2 by using group information from DescribeGroupsRequest. This means that we need to have Consumer Group stopped before executing and start a new consumer internally to do this? Therefore, we won't be able to consider executing reset when ConsumerGroup is active? (trying to relate it with @Dong 5th question) @Dong: > Should we allow user to use wildcard to reset offset of all groups for a given topic as well? I haven't thought about this scenario. Could be interesting. Following the recommendation to add it into Consumer Group Command, in this case Group argument will be optional if there are only 1 topic. I think for multiple topic won't be that useful. > Should we allow user to specify timestamp per topic partition in the json file as well? Don't think this could be a valid from the tool, but if Reset Plan is generated, and user want to set the offset for a specific partition to other offset (eventually based on another timestamp), and execute it, it will be up to her/him. > Should the script take some credential file to make sure that this operation is authenticated given the potential impact of this operation? Haven't tried to secure brokers yet, but the tool should support authorization if it's enabled in the broker. > Should we provide constant to reset committed offset to earliest/latest offset of a partition, e.g. -1 indicates earliest offset and -2 indicates latest offset. I will go for something like ´--reset-to-earliest´ and ´--reset-to-latest´ > Should we allow dynamic change of the comitted offset when consumer are running, such that consumer will seek to the newly committed offset and start consuming from there? Not sure about this. I will recommend to keep it simple and ask user to stop consumers first. But I would considered it if the trade-offs are clear. @Matthias Added :). And thanks a lot for your help to define this KIP! El mié., 8 feb. 2017 a las 7:47, Gwen Shapira (<g...@confluent.io>) escribió: > As long as the CLI is a bit consistent? Like, not just adding 3 > arguments and a JSON parser to the existing tool, right? > > On Tue, Feb 7, 2017 at 10:29 PM, Onur Karaman > <onurkaraman.apa...@gmail.com> wrote: > > I think it makes sense to just add the feature to > kafka-consumer-groups.sh > > > > On Tue, Feb 7, 2017 at 10:24 PM, Gwen Shapira <g...@confluent.io> wrote: > > > >> Thanks for the KIP. I'm super happy about adding the capability. > >> > >> I hate the interface, though. It looks exactly like the replica > >> assignment tool. A tool everyone loves so much that there are multiple > >> projects, open and closed, that try to fix it. > >> > >> Can we swap it with something that looks a bit more like the consumer > >> group tool? or the kafka streams reset tool? Consistency is helpful in > >> such cases. I spent some time learning existing tools and learning yet > >> another one is a deterrent. > >> > >> Gwen > >> > >> > >> > >> On Tue, Feb 7, 2017 at 6:43 PM, Jorge Esteban Quilcate Otoya > >> <quilcate.jo...@gmail.com> wrote: > >> > Hi all, > >> > > >> > I would like to propose a KIP to Add a tool to Reset Consumer Group > >> Offsets. > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >> 122%3A+Add+a+tool+to+Reset+Consumer+Group+Offsets > >> > > >> > Please, take a look at the proposal and share your feedback. > >> > > >> > Thanks, > >> > Jorge. > >> > >> > >> > >> -- > >> Gwen Shapira > >> Product Manager | Confluent > >> 650.450.2760 <(650)%20450-2760> | @gwenshap > >> Follow us: Twitter | blog > >> > > > > -- > Gwen Shapira > Product Manager | Confluent > 650.450.2760 <(650)%20450-2760> | @gwenshap > Follow us: Twitter | blog >
