Hi Bruno, Thanks for the update! Hi Boyuan, For listTopics() method, it'll *always* do a remote call, which will have performance impact for sure. For partitionsFor() method, it'll *check cache first*, if not found in cache, then do a remote call to retrieve the topic partition info.
So, I think partitionsFor() should be a better option for you. Thanks. Luke On Fri, Jan 8, 2021 at 2:38 AM Boyuan Zhang <boyu...@apache.org> wrote: > Thanks, folks! > > It seems like partitionsFor() and listTopics() is what I want. Do we have > performance estimates on these 2 API calls, e.g., the time cost of waiting > for responses? I would invoke these API along a hot path so I want to have > a general idea on how bad it could be. > > Many thanks to your help! > > On Thu, Jan 7, 2021 at 1:44 AM Bruno Cadonna <br...@confluent.io> wrote: > > > Hi Luke, > > > > I am afraid the ConsumerRebalanceListener will not work in this case > > since Boyuan assigns the partitions manually. The Java docs you linked > > state > > > > If the consumer directly assigns partitions, those partitions will never > > be reassigned and this callback is not applicable. > > > > > > Hi Boyuan, > > > > The consumer has methods partitionsFor() and listTopics(). Probably > > there is a better way to get the information you want that I am not > > aware of. > > > > Best, > > Bruno > > > > On 07.01.21 05:09, Luke Chen wrote: > > > Hi Boyuan, > > > You can create a *ConsumerRebalanceListener* and do something you want > > when > > > *onPartitionsRevoked. * > > > Please check this java doc for more information: > > > > > > https://kafka.apache.org/27/javadoc/org/apache/kafka/clients/consumer/ConsumerRebalanceListener.html > > > > > > Thanks. > > > Luke > > > > > > On Thu, Jan 7, 2021 at 8:45 AM Boyuan Zhang <boyu...@apache.org> > wrote: > > > > > >> Hi team, > > >> > > >> I'm working on a long run application, which uses the Kafka Consumer > > API to > > >> poll messages from a given topic and partition. I'm assigning the > topic > > and > > >> partition manually by using consumer.assign() API and polling messages > > by > > >> using consumer.poll(). > > >> > > >> One common scenario for my application is that certain partitions > could > > be > > >> removed outside of my application and my application needs to know one > > >> partition has been removed to stop processing that partition. My > > question > > >> is that is there any way to get the removal information when I do > > >> consumer.assign() or consumer.poll() or any APIs that I can use? > > >> > > >> Thanks for your help! > > >> > > > > > >
