Thanks Neha, So are you saying that on 0.7.2, to delete a topic I need only remove it's data log directory from each broker, and the restart the brokers? Is it ok if it's a rolling restart?
For some reason I thought I also had to do something on zookeeper servers too, to remove the topics. Is that not true? Jason On Wed, Mar 6, 2013 at 6:07 PM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > >> The topics get automagically created if they don't exist. Will > that still exist in 0.8? > > In 0.8, this is controlled by the auto.create.topics.enable config on the > brokers. If this is set to true, topics will be created when a topic > metadata request is sent for a new topic. This feature is provided to aid > with the 0.7->0.8 migration tool and the mirror maker tools. These tools > copy data for almost all topics from one cluster to the other, so auto > creating topics makes it much easier to migrate data between clusters. > > >> What's the procedure for deleting topics in kafka 0.7.2? > > There is no official way to delete a topic. You have to delete the data > directory and bounce the brokers to take note of that. > > Thanks, > Neha > > > On Wed, Mar 6, 2013 at 4:20 PM, Jason Rosenberg <j...@squareup.com> wrote: > > > Thanks Neha, > > > > I added a comment to that ticket. > > > > Another question, I'm not sure if the behavior is different in 0.8 from > > 0.7. But currently, I'm not predeclaring topics, before sending messages > > to topics. The topics get automagically created if they don't exist. > Will > > that still exist in 0.8? If so, it should also be automatic (using a > > default expiration for all such topics), that they automagically go away > > after no longer being in use. > > > > What's the procedure for deleting topics in kafka 0.7.2? > > > > Jason > > > > On Wed, Mar 6, 2013 at 3:38 PM, Neha Narkhede <neha.narkh...@gmail.com > > >wrote: > > > > > We plan to include support for deleting topics before the official > > release. > > > The JIRA tracking this feature is > > > https://issues.apache.org/jira/browse/KAFKA-330 > > > > > > This is mainly an admin tool and is proposed to be run manually. I > think > > > your suggestion is to set some sort of expiration time on topics so > when > > > all data gets garbage collected and the expiration time lapses, the > topic > > > is automatically deleted. > > > > > > For this expiration to work, we need some mechanism of knowing the size > > of > > > the data logs for that topic on all brokers. And only when its empty > for > > > all partitions of that topic, we delete it. This requires some API that > > > will expose the data size of a topic on a broker, which we don't have > > right > > > now. It might be worth thinking about it. Do you mind posting your > > > suggestions for this tool on that JIRA ? > > > > > > Thanks, > > > Neha > > > > > > > > > On Wed, Mar 6, 2013 at 3:15 PM, Jason Rosenberg <j...@squareup.com> > > wrote: > > > > > > > I suspect this is not currently supported, but it seems to be for us > a > > > real > > > > use case. > > > > > > > > If we have a topic that is no longer receiving messages, and all > > messages > > > > have been removed from the brokers, after the log_retention_hours has > > > > expired, I'd love to have it then automatically remove the topic from > > > > kafka. > > > > > > > > If subsequently a new message arrives for that topic, it could just > > > > recreate the topic as originally, and continue on. > > > > > > > > Is this easy to implement? > > > > Has it been requested? > > > > Does it make sense? > > > > Should I file a jira? > > > > > > > > Currently, we had a test client that inadvertently created 1000's of > > > > topics. The job of deleting all these appears to not be a cumbersome > > > > manual process (are there any quick links to the recommended way for > > > doing > > > > this?). > > > > > > > > I am using 0.7.2, but will eventually upgrade to 0.8, once it's > > released > > > to > > > > beta.... > > > > > > > > Thanks, > > > > > > > > Jason > > > > > > > > > >
