AndrewJSchofield commented on code in PR #17626: URL: https://github.com/apache/kafka/pull/17626#discussion_r1824208417
########## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ########## @@ -3490,6 +3490,138 @@ void handleFailure(Throwable throwable) { return new DescribeDelegationTokenResult(tokensFuture); } + private static final class ListGroupsResults { + private final List<Throwable> errors; + private final HashMap<String, GroupListing> listings; + private final HashSet<Node> remaining; + private final KafkaFutureImpl<Collection<Object>> future; + + ListGroupsResults(Collection<Node> leaders, + KafkaFutureImpl<Collection<Object>> future) { + this.errors = new ArrayList<>(); + this.listings = new HashMap<>(); + this.remaining = new HashSet<>(leaders); + this.future = future; + tryComplete(); + } + + synchronized void addError(Throwable throwable, Node node) { + ApiError error = ApiError.fromThrowable(throwable); + if (error.message() == null || error.message().isEmpty()) { + errors.add(error.error().exception("Error listing groups on " + node)); + } else { + errors.add(error.error().exception("Error listing groups on " + node + ": " + error.message())); + } + } + + synchronized void addListing(GroupListing listing) { + listings.put(listing.groupId(), listing); + } + + synchronized void tryComplete(Node leader) { + remaining.remove(leader); + tryComplete(); + } + + private synchronized void tryComplete() { + if (remaining.isEmpty()) { + ArrayList<Object> results = new ArrayList<>(listings.values()); + results.addAll(errors); + future.complete(results); + } + } + } + + @Override + public ListGroupsResult listGroups(ListGroupsOptions options) { + final KafkaFutureImpl<Collection<Object>> all = new KafkaFutureImpl<>(); + final long nowMetadata = time.milliseconds(); + final long deadline = calcDeadlineMs(nowMetadata, options.timeoutMs()); + runnable.call(new Call("findAllBrokers", deadline, new LeastLoadedNodeProvider()) { + @Override + MetadataRequest.Builder createRequest(int timeoutMs) { + return new MetadataRequest.Builder(new MetadataRequestData() + .setTopics(Collections.emptyList()) + .setAllowAutoTopicCreation(true)); + } + + @Override + void handleResponse(AbstractResponse abstractResponse) { + MetadataResponse metadataResponse = (MetadataResponse) abstractResponse; + Collection<Node> nodes = metadataResponse.brokers(); + if (nodes.isEmpty()) + throw new StaleMetadataException("Metadata fetch failed due to missing broker list"); + + HashSet<Node> allNodes = new HashSet<>(nodes); + final ListGroupsResults results = new ListGroupsResults(allNodes, all); + + for (final Node node : allNodes) { + final long nowList = time.milliseconds(); + runnable.call(new Call("listGroups", deadline, new ConstantNodeIdProvider(node.id())) { + @Override + ListGroupsRequest.Builder createRequest(int timeoutMs) { + List<String> groupTypes = options.types() + .stream() + .map(GroupType::toString) + .collect(Collectors.toList()); + return new ListGroupsRequest.Builder(new ListGroupsRequestData() + .setTypesFilter(groupTypes) + ); + } + + private void maybeAddGroup(ListGroupsResponseData.ListedGroup group) { + final String groupId = group.groupId(); + final GroupType type = group.groupType().isEmpty() + ? GroupType.UNKNOWN + : GroupType.parse(group.groupType()); + final String protocolType = group.protocolType(); + final GroupListing groupListing = new GroupListing( + groupId, + Optional.of(type), Review Comment: This new code is part of AK 4.0. If you use this new admin client to list groups for a broker which does not support ListGroups v5 or later, it cannot return the group type. In this case, I think the `GroupListing` should use `Optional.empty()` for the group type. I don't have a problem with `ListGroupsOptions.withTypes` not being able to request information on groups whose type cannot be returned. Using a set of `Optional` is cumbersome for a situation which is unlikely to be used in practice. If you use this admin client to list groups for a future broker that has more group types than this admin client understands, I think the `GroupListing` should use `Optional.of(GroupType.UNKNOWN)` for the group type. I think it's very unlikely that a user would want to query groups whose type cannot be understood. Technically, `GroupType.UNKNOWN` exists so it could be supplied in the `withTypes` method. It's not a sensible thing to attempt to do. I wonder whether the best thing would be to ensure that `GroupType.UNKNOWN` is not specified in the set of group types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org