Hi Aditya, Thanks for your reply. I'm not dismissing your concern, but I'm not entirely happy with a way forward yet.
AK1: I wonder whether we could use a pipe here to avoid the intermediate file. After all, export uses stdout as the destination, so if we could use stdin as the source. For example, I've used '-' with --from-file below to indicate stdin. It could be --from-stdin or similar too. $ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --export-offsets --group CG1 --topic T1 --to-current | bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --reset-offsets --group CG2 --from-file - --topic T1 --execute This would still allow resetting between different group types, which is actually the use case that got me into this, taking offsets from a consumer group to initialise a share group. Thanks, Andrew On 2026/08/12 18:12:08 Aditya Kousik wrote: > Hi Andrew. > > Thanks for addressing AK2-4: lgtm > > On AK1: I’m thinking from a user experience standpoint. The teams managing > the kafka clusters and the ones using the clients diverge - but the Admin > role may be exposed via self-service typically via API requests and a K8S > CRD. I’m thinking of the latter case where Kafka SRE allow a thin wrapper for > client teams to manage their own groups and topics and also isolating the > environment where the scripts can be run. In such a case, it becomes > cumbersome to fetch/write the csv file from a Pod that runs the migration ; > it may not have any volume to persist it or running kubectl to scp the file > is usually denied. So in such a case, the migrate offsets script could just > be a single command that fetches offsets from one group and applies it to > another (or several) without the middleman CSV. > > Thanks, > Aditya > > > On Aug 12, 2026, at 06:39, Andrew Schofield <[email protected]> wrote: > > > > Hi Aditya, > > Thanks for your reply. Sorry for the delay. > > > > AK1: One of the advantages of the file is that it's possible to export from > > one group type and import into another. You can also export offsets for > > multiple groups, and this changes the file format (4 columns instead of 3). > > Sadly, this is all existing behaviour that I only learnt by trial and > > error. Personally, I think that export-import is a relatively infrequent > > operation, and the KIP makes enough of a step forward. I wonder if you > > could elaborate a bit more on your idea. Feel free to push back if you > > disagree. > > > > AK2: The KIP does not mention this and I wasn't quite sure where to add it. > > You are correct. Should we improve the javadoc for the admin client? > > > > AK3: Yes, Admin is better. KIP updated. > > > > AK4: The DescribeGroupsGeneric RPC is intended as a way of getting a > > minimal set of group information for groups of any group types. It is used > > in Admin.describeGroups and this would be the initial Admin call from > > bin/kafka-XXXX-groups.sh when exporting or resetting offsets. It gives a > > way to check the group type and existence unambiguously. By contrast, if > > you use Admin.describeXXXGroups() for an existing group ID of the wrong > > type, the error code is GROUP_ID_NOT_FOUND. > > > > AK4.1: I didn't include the members intentionally. This RPC is agnostic to > > group type. I think the common subset of group attributes is group ID, > > group type, group state and coordinator. If you dig deeper, you find that > > the group types differ. For example, assignments in consumer groups and > > share groups are partitions, while in streams groups they are tasks. I > > expect we'll add connect groups one day, and until we have the KIP, I > > wouldn't want to assume the details of membership. > > > > AK4.2: This is following existing convention I'm afraid and I want to be > > consistent. If the broker does not send back a list of ACL operations, then > > this will be null. This is different than an empty set which means "you are > > not authorized", as opposed to "you don't know anything about authorized > > operations". > > > > Thanks, > > Andrew > > > >> On 2026/07/27 05:45:27 Aditya Kousik wrote: > >> A small correction. > >> > >> I meant to add a statement before the numbered comments. > >> > >> “Since the KIP motivation revolves around usability of the script, would > >> you be willing to consider the following:” > >> > >> Also, typo on AK2. “…As long as there aren’t any active members in the > >> group subscription”. > >> > >> -Aditya > >> > >>>> On Jul 25, 2026, at 23:53, Aditya Kousik <[email protected]> wrote: > >>> > >>> Hi Andrew, > >>> > >>> Thanks for this KIP, I’ve personally faced more than one scenario where > >>> I’ve had to use the admin client to update the offsets. > >>> > >>> The main uses cases teams face ( and TIL about the various options the > >>> script offers): > >>> Teams migrating to a new consumer group from an existing one but don’t > >>> want to commit to either (or duration for newer clients) auto offset > >>> reset policy on the new group. > >>> Teams want to seek past certain offsets in partitions because they have > >>> poison pill messages/corrupted deserialization story and they have no > >>> control over the data. In this case, they actually want latest offset > >>> strategy but most don’t even realise that it applies to streams out of > >>> range and are surprised/frustrated that the property doesn’t move their > >>> position in the queues. (I see we can do this already in the script today) > >>> When I migrated our clients from 2.x directly to 4.x, out of abundance of > >>> caution, I wrote an hourly cron job that dumps the snapshot of > >>> group_id,topic,partition,offset into csv files so that if I had to revert > >>> a few affected partitions I can trim the csv down to those partitions and > >>> alter them with the admin client. > >>> > >>> AK1. A single command line with args —source-group-id —target-group-id > >>> —export so that the intermediate CSV file can be skipped. > >>> AK2. Minor: The KIP does not mention the alter/list consumer group > >>> offsets Admin call. alter only works as long as there are active members > >>> in the group subscription. I was bitten by this in the past; the > >>> documentation was sparse around that expectation but it does make sense. > >>> AK3. KIP references AdminClient but I believe this is deprecated in > >>> favour of just Admin which is the interface that has the methods? > >>> AK4. How do we leverage the describe groups RPC when using the > >>> script/admin client? I’m trying to imagine flows other than the export > >>> group offsets. > >>> AK4.1 Can we list active members/offset state in a given group in > >>> addition to the GroupState? This has helped identify zombie processes to > >>> hunt down. The broker’s view would be useful here. (related to AK2) > >>> AK4.2 Could we return empty set of acl operations instead of null? > >>> > >>> Best, > >>> Aditya > >>> > >>> > >>>> On 2026/06/17 06:34:11 Andrew Schofield wrote: > >>>> Hi, > >>>> I’d like to start discussion on a small KIP to improve the usability of > >>>> the command-line tools for resetting group offsets. When implementing > >>>> KIP-1323 and reviewing the PRs, it became clear that the usability of > >>>> this area of Kafka isn’t great. This KIP improves the usability of these > >>>> tools. > >>>> > >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1359%3A+Improve+usability+of+resetting+group+offsets > >>>> > >>>> Thanks, > >>>> Andrew > >>>> > >> >
