I'd like just to add that in my specific case I could resolve creating a new NodeProvider implementation like LeaderForProvider in order to send the request to the right leader for each topic.
The drawback of this solution is not grouping topics for the same leader so not having only one requests but different of them (even if for the same leader). Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno<http://twitter.com/ppatierno> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> Blog : DevExperience<http://paolopatierno.wordpress.com/> ________________________________ From: Paolo Patierno <ppatie...@live.com> Sent: Monday, October 23, 2017 9:06 AM To: dev@kafka.apache.org Subject: Metadata class doesn't "expose" topics with errors Hi devs, while developing the KIP-204 (having delete records operation in the "new" Admin Client) I'm facing with the following doubt (or maybe a lack of info) ... As described by KIP-107 (which implements this feature at protocol level and in the "legacy" Admin Client), the request needs to be sent to the leader. For both KIPs, the operation has a Map<TopicPartition, offset> (offset is a long in the "legacy" API but it's becoming to be a class in the "new" API) and in order to reduce the number of requests to different leaders, my code groups partitions having same leader so having a Map<Node, Map<TopicPartition, offset>>. In order to know the leaders I need to request metadata and there are two ways for doing that : * using something like the producer does with Metadata class, putting the topics, request update and waiting for it * using the low level MetadataRequest and handling the related response (which is what the "legacy" API does today) I noticed that building the Cluster object from the MetadataResponse, the topics with errors are skipped and it means that in the final "high level" Metadata class (fetching the Cluster object) there is no information about them. So with first solution we have no info about topics with errors (maybe the only errors I'm able to handle is the "LEADER_NOT_AVAILABLE", if leaderFor() on the Cluster returns a null Node). Is there any specific reason why "topics with errors" are not exposed in the Metadata instance ? Is the preferred pattern using the low level protocol stuff in such case ? Thanks Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno<http://twitter.com/ppatierno> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> Blog : DevExperience<http://paolopatierno.wordpress.com/>
