Hi Viktor, Thanks for bumping this thread.
I think we should just focus on transitioning the TopicCommand to use AdminClient, and talk about protocol changes in a separate KIP. Protocol changes often involve a lot of discussion. This does mean that we couldn't implement the "list topics under deletion" feature when using AdminClient at the moment. We could add a note to the tool output indicating this. We should move the protocol discussion to a separate thread. Probably also look at KIP-142 as well. best, Colin On Tue, Oct 9, 2018, at 07:45, Viktor Somogyi-Vass wrote: > Hi All, > > Would like to bump this as the conversation sank a little bit, but more > importantly I'd like to validate my plans/ideas on extending the Metadata > protocol. I was thinking about two other alternatives, namely: > 1. Create a ListTopicUnderDeletion protocol. This however would be > unnecessary: it'd have one very narrow functionality which we can't extend. > I'd make sense to have a list topics or describe topics protocol where we > can list/describe topics under deletion but for normal listing/describing > we already use the metadata, so it would be a duplication of functionality. > 2. DeleteTopicsResponse could return the topics under deletion if the > request's argument list is empty which might make sense at the first look, > but actually we'd mix the query functionality with the delete functionality > which is counterintuitive. > > Even though most clients won't need these "limbo" topics (which are under > deletion) in the foreseeable future, it can be considered as part of the > cluster state or metadata and to me it makes sense. Also it doesn't have a > big overhead in the response size as typically users don't delete topics > too often as far as I experienced. > > I'd be happy to receive some ideas/feedback on this. > > Cheers, > Viktor > > > On Fri, Sep 28, 2018 at 4:51 PM Viktor Somogyi-Vass <viktorsomo...@gmail.com> > wrote: > > > Hi All, > > > > I made an update to the KIP. Just in short: > > Currently KafkaAdminClient.describeTopics() and > > KafkaAdminClient.listTopics() uses the Metadata protocol to acquire topic > > information. The returned response however won't contain the topics that > > are under deletion but couldn't complete yet (for instance because of some > > replicas offline), therefore it is not possible to implement the current > > command's "marked for deletion" feature. To get around this I introduced > > some changes in the Metadata protocol. > > > > Thanks, > > Viktor > > > > On Fri, Sep 28, 2018 at 4:48 PM Viktor Somogyi-Vass < > > viktorsomo...@gmail.com> wrote: > > > >> Hi Mickael, > >> > >> Thanks for the feedback, I also think that many customers wanted this for > >> a long time. > >> > >> Cheers, > >> Viktor > >> > >> On Fri, Sep 28, 2018 at 11:45 AM Mickael Maison <mickael.mai...@gmail.com> > >> wrote: > >> > >>> Hi Viktor, > >>> Thanks for taking this task! > >>> This is a very nice change as it will allow users to use this tool in > >>> many Cloud environments where direct zookeeper access is not > >>> available. > >>> > >>> > >>> On Thu, Sep 27, 2018 at 10:34 AM Viktor Somogyi-Vass > >>> <viktorsomo...@gmail.com> wrote: > >>> > > >>> > Hi All, > >>> > > >>> > This is the continuation of the old KIP-375 with the same title: > >>> > > >>> https://lists.apache.org/thread.html/dc71d08de8cd2f082765be22c9f88bc9f8b39bb8e0929a3a4394e9da@%3Cdev.kafka.apache.org%3E > >>> > > >>> > The problem there was that two KIPs were created around the same time > >>> and I > >>> > chose to reorganize mine a bit and give it a new number to avoid > >>> > duplication. > >>> > > >>> > The KIP summary here once again: > >>> > > >>> > I wrote up a relatively simple KIP about improving the Kafka protocol > >>> and > >>> > the TopicCommand tool to support the new Java based AdminClient and > >>> > hopefully to deprecate the Zookeeper side of it. > >>> > > >>> > I would be happy to receive some opinions about this. In general I > >>> think > >>> > this would be an important addition as this is one of the few left but > >>> > important tools that still uses direct Zookeeper connection. > >>> > > >>> > Here is the link for the KIP: > >>> > > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-377%3A+TopicCommand+to+use+AdminClient > >>> > > >>> > Cheers, > >>> > Viktor > >>> > >>
