Yeah Joel, I just thought of that idea too (e.g. temporarily disable auto topic creation). I can probably make that work (it may result in unexpected lack of topic creation, but I can message this as a maintenance downtime, etc.).
So, that was my next question, so a consumer rebalance will trigger a fresh topic pull from the consumers? How long is 'long enough' to ensure a rebalance has occurred everywhere? Jason On Mon, Jan 26, 2015 at 3:07 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > Hey Jason, > > Is it an option for you to do the following: > > - Bounce in a config change to the brokers to turn off auto-create > - (Batch)-delete the topic(s) > - Wait long enough for consumers to rebalance (after which they will > no longer consume the topic(s)) > - Bounce in a config change to the brokers to turn on auto-create > > Joel > > On Mon, Jan 26, 2015 at 12:01:59PM -0800, Jun Rao wrote: > > Hi, Jason, > > > > I am not sure that we should delay the 0.8.2 release. The reasons are (1) > > There are other features such as the new java producer and Kafka-based > > offset management that are potentially useful to people. (2) It may take > > some time to completely fix the issue with deleting topic since it > involves > > wire protocol changes. > > > > What we can do is probably just to document the limitation of delete > topic > > in 0.8.2.0 in our site. > > > > Thanks, > > > > Jun > > > > On Mon, Jan 26, 2015 at 11:05 AM, Jason Rosenberg <j...@squareup.com> > wrote: > > > > > I think this shortcoming should probably delay release of 0.8.2 until > > > resolved, no? > > > > > > On Mon, Jan 26, 2015 at 2:01 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > > > It then seems to me that delete-topic will not actually work > "smoothly" > > > > until create topic request is added since it is too much to require > > > people > > > > to turn off their clients while deleting topics. In this case shall > we > > > make > > > > it clear in the release docs of in 0.8.2, or even still mark it as > > > > not-supported? > > > > > > > > Guozhang > > > > > > > > On Mon, Jan 26, 2015 at 8:14 AM, Harsha <ka...@harsha.io> wrote: > > > > > > > > > Jun, > > > > > I made an attempt at fixing that issue as part of this JIRA > > > > > https://issues.apache.org/jira/browse/KAFKA-1507 . > > > > > As Jay pointed out there should be admin api if there is more info > on > > > > > this api I am interested in adding/fixing this issue. > > > > > Thanks, > > > > > Harsha > > > > > > > > > > On Mon, Jan 26, 2015, at 07:28 AM, Jun Rao wrote: > > > > > > Yes, that's the issue. Currently, topics can be auto-created on > > > > > > TopicMetadataRequest, which can be issued from both the producer > and > > > > the > > > > > > consumer. To prevent that, you would need to stop the producer > and > > > the > > > > > > consumer before deleting a topic. We plan to address this issue > once > > > we > > > > > > have a separate request for creating topics. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Jun > > > > > > > > > > > > On Mon, Jan 26, 2015 at 7:21 AM, Harsha <ka...@harsha.io> wrote: > > > > > > > > > > > > > There could be another case where if you have > > > > auto.create.topics.enable > > > > > > > to set to true ( its true by default) . Any > TopicMetadataRequest > > > can > > > > > > > recreate topics. So if you issued a delete topic command and > you > > > have > > > > > > > producers running or consumers? too which is issuing a > > > > > > > TopicMetadataRequest than the topic will be recreated. > > > > > > > -Harsha > > > > > > > > > > > > > > On Sun, Jan 25, 2015, at 11:26 PM, Jason Rosenberg wrote: > > > > > > > > cversion did change (incremented by 2) when I issue the > delete > > > > > command. > > > > > > > > > > > > > > > > From the logs on the conroller broker (also the leader for > the > > > > > topic), it > > > > > > > > looks like the delete proceeds, and then the topic gets > recreated > > > > > > > > immediately (highlighted in yellow). It appears maybe it’s > due > > > to a > > > > > > > > consumer client app trying to consume the topic. Also, the > > > consumer > > > > > is > > > > > > > > not > > > > > > > > yet updated to 0.8.2 (it’s using 0.8.1.1), perhaps that’s > part of > > > > the > > > > > > > > problem? > > > > > > > > > > > > > > > > > > > > > > > > 2015-01-26 07:02:14,281 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.PartitionStateMachine$DeleteTopicsListener - > > > > > > > > [DeleteTopicsListener on 6]: Starting topic deletion for > topics > > > > > > > > mytopic > > > > > > > > 2015-01-26 07:02:14,282 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Handling deletion for topics > mytopic > > > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic > (re)started > > > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager - [Topic Deletion Manager 6], > > > Topic > > > > > > > > deletion callback for mytopic > > > > > > > > 2015-01-26 07:02:14,289 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager - [Topic Deletion Manager 6], > > > > > > > > Partition deletion callback for [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,295 INFO [delete-topics-thread-6] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to OfflineReplica for replicas > > > > > > > > > > > > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > > > 2015-01-26 07:02:14,303 INFO [delete-topics-thread-6] > > > > > > > > controller.KafkaController - [Controller 6]: New leader and > ISR > > > for > > > > > > > > partition [mytopic,0] is > {"leader":6,"leader_epoch":1,"isr":[6]} > > > > > > > > 2015-01-26 07:02:14,312 INFO [delete-topics-thread-6] > > > > > > > > controller.KafkaController - [Controller 6]: New leader and > ISR > > > for > > > > > > > > partition [mytopic,0] is > {"leader":-1,"leader_epoch":2,"isr":[]} > > > > > > > > 2015-01-26 07:02:14,313 INFO [delete-topics-thread-6] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to ReplicaDeletionStarted for > replicas > > > > > > > > > > > > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-5] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 6] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 6] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > > > > > > log.OffsetIndex - Deleting index > > > > > > > > /mypath/mytopic-0/00000000000000000000.index > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7] > > > > > log.LogManager > > > > > > > > - Deleted log for partition [mytopic,0] in /mypath/mytopic-0. > > > > > > > > 2015-01-26 07:02:14,314 INFO > > > > [Controller-6-to-broker-6-send-thread] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful for > > > replicas > > > > > > > > [Topic=mytopic,Partition=0,Replica=6] > > > > > > > > 2015-01-26 07:02:14,314 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Handling deletion for topics > mytopic > > > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Deletion for replicas 7 for > partition > > > > > > > > [mytopic,0] of topic mytopic in progress > > > > > > > > 2015-01-26 07:02:14,316 INFO > > > > [Controller-6-to-broker-7-send-thread] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful for > > > replicas > > > > > > > > [Topic=mytopic,Partition=0,Replica=7] > > > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Handling deletion for topics > mytopic > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to NonExistentReplica for replicas > > > > > > > > > > > > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=6],[Topic=mytopic,Partition=0,Replica=7] > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > > > controller.PartitionStateMachine - [Partition state machine > on > > > > > > > > Controller 6]: Invoking state change to OfflinePartition for > > > > > > > > partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6] > > > > > > > > controller.PartitionStateMachine - [Partition state machine > on > > > > > > > > Controller 6]: Invoking state change to NonExistentPartition > for > > > > > > > > partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,325 WARN [kafka-request-handler-2] > > > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]: Fetch > > > > request > > > > > > > > with correlation id 2619442 from client > ReplicaFetcherThread-0-6 > > > on > > > > > > > > partition [mytopic,0] failed due to Partition [mytopic,0] > doesn't > > > > > > > > exist on 6 > > > > > > > > 2015-01-26 07:02:14,371 INFO [delete-topics-thread-6] > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread - > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic > successfully > > > > > > > > completed > > > > > > > > 2015-01-26 07:02:14,376 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.PartitionStateMachine$TopicChangeListener - > > > > > > > > [TopicChangeListener on Controller 6]: New topics: [Set()], > > > deleted > > > > > > > > topics: [Set()], new partition replica assignment [Map()] > > > > > > > > 2015-01-26 07:02:14,403 WARN [request-expiration-task] > > > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]: Fetch > > > > request > > > > > > > > with correlation id 2054626 from client > > > > > > > > myconsumer-1415333962444-cc78e67e-0-6 on partition > [mytopic,0] > > > > failed > > > > > > > > due to Partition [mytopic,0] doesn't exist on 6 > > > > > > > > 2015-01-26 07:02:14,446 INFO [kafka-request-handler-5] > > > > > > > > admin.AdminUtils$ - Topic creation > > > > > > > > {"version":1,"partitions":{"0":[7,6]}} > > > > > > > > 2015-01-26 07:02:14,457 INFO [kafka-request-handler-5] > > > > > > > > server.KafkaApis - [KafkaApi-6] Auto creation of topic > mytopic > > > > with 1 > > > > > > > > partitions and replication factor 2 is successful! > > > > > > > > 2015-01-26 07:02:14,458 INFO [kafka-network-thread-27330-2] > > > > > > > > network.Processor - Closing socket connection to /10.1.10.4. > > > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.PartitionStateMachine$TopicChangeListener - > > > > > > > > [TopicChangeListener on Controller 6]: New topics: > > > [Set(mytopic)], > > > > > > > > deleted topics: [Set()], new partition replica assignment > > > > > > > > [Map([mytopic,0] -> List(7, 6))] > > > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.KafkaController - [Controller 6]: New topic > creation > > > > > > > > callback for [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.KafkaController - [Controller 6]: New partition > > > creation > > > > > > > > callback for [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,466 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.PartitionStateMachine - [Partition state machine > on > > > > > > > > Controller 6]: Invoking state change to NewPartition for > > > partitions > > > > > > > > [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,467 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to NewReplica for replicas > > > > > > > > > > > > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > > > 2015-01-26 07:02:14,468 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.PartitionStateMachine - [Partition state machine > on > > > > > > > > Controller 6]: Invoking state change to OnlinePartition for > > > > > partitions > > > > > > > > [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,530 INFO > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace] > > > > > > > > controller.ReplicaStateMachine - [Replica state machine on > > > > controller > > > > > > > > 6]: Invoking state change to OnlineReplica for replicas > > > > > > > > > > > > > > > > > > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6] > > > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-5] > log.Log - > > > > > > > > Completed load of log mytopic-0 with log end offset 0 > > > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-5] > > > > > log.LogManager > > > > > > > > - Created log for partition [mytopic,0] in /mypath with > > > properties > > > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms -> > 60000, > > > > > > > > segment.bytes -> 1073741824, flush.ms -> > 9223372036854775807, > > > > > > > > delete.retention.ms -> 86400000, index.interval.bytes -> > 4096, > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas -> 1, > > > > > > > > cleanup.policy -> delete, unclean.leader.election.enable -> > true, > > > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012, > > > > > flush.messages > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, > > > > > retention.ms > > > > > > > > -> 86400000, segment.jitter.ms -> 0}. > > > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 6] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5] > log.Log - > > > > > > > > Truncating log mytopic-0 to offset 0. > > > > > > > > 2015-01-26 07:02:14,547 INFO [kafka-request-handler-5] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 6] > > > > > > > > Added fetcher for partitions List([[mytopic,0], initOffset 0 > to > > > > > broker > > > > > > > > id:7,host:mybroker:54321] ) > > > > > > > > > > > > > > > > Here’s the log in the other broker with a replica of the > topic. > > > > > > > > > > > > > > > > 2015-01-26 07:02:14,312 INFO [kafka-request-handler-7] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 7] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 7] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > > > > > > log.OffsetIndex - Deleting index > > > > > > > > /mypath/mytopic-0/00000000000000000000.index > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2] > > > > > log.LogManager > > > > > > > > - Deleted log for partition [mytopic,0] in /mypath/mytopic-0. > > > > > > > > 2015-01-26 07:02:14,530 INFO [kafka-request-handler-0] > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager on > broker > > > 7] > > > > > > > > Removed fetcher for partitions [mytopic,0] > > > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-0] > log.Log - > > > > > > > > Completed load of log mytopic-0 with log end offset 0 > > > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-0] > > > > > log.LogManager > > > > > > > > - Created log for partition [mytopic,0] in /mypath with > > > properties > > > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms -> > 60000, > > > > > > > > segment.bytes -> 1073741824, flush.ms -> > 9223372036854775807, > > > > > > > > delete.retention.ms -> 86400000, index.interval.bytes -> > 4096, > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas -> 1, > > > > > > > > cleanup.policy -> delete, unclean.leader.election.enable -> > true, > > > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012, > > > > > flush.messages > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, > > > > > retention.ms > > > > > > > > -> 86400000, segment.jitter.ms -> 0}. > > > > > > > > > > > > > > > > On Mon, Jan 26, 2015 at 1:26 AM, Jun Rao <j...@confluent.io> > > > wrote: > > > > > > > > > > > > > > > > Could you do a get on /admin/deleted_topics and see if its > > > cversion > > > > > > > > changes > > > > > > > > > after issuing the delete topic command? This will tell us > > > whether > > > > > the > > > > > > > > > path /admin/deleted_topics//mytopic > > > > > > > > > is actually created or not. Anything in the controller log > that > > > > > > > indicates a > > > > > > > > > delete topic event is fired? > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 9:55 PM, Jason Rosenberg < > > > > j...@squareup.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > yes > > > > > > > > > > > > > > > > > > > > On Mon, Jan 26, 2015 at 12:18 AM, Jun Rao < > j...@confluent.io> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Do you have delete.topic.enable turned on in all > brokers? > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 7:56 PM, Jason Rosenberg < > > > > > j...@squareup.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > So far, I have been unable to get delete topic to > work, > > > > with > > > > > > > release > > > > > > > > > > > > candidate 2 for 0.8.2. > > > > > > > > > > > > > > > > > > > > > > > > It worked ok when I ran it in the debugger locally, > on a > > > > > single > > > > > > > node > > > > > > > > > > > > instance. But when I run it in our staging > environment, > > > it > > > > > is not > > > > > > > > > > > > successfully even marking the topic for delete, for > some > > > > > reason. > > > > > > > > > > > > > > > > > > > > > > > > I am setting delete.topic.enable to true (and see the > > > > > > > confirmation of > > > > > > > > > > > this > > > > > > > > > > > > in the startup logs, e.g.): > > > > > > > > > > > > > > > > > > > > > > > > INFO [main] utils.VerifiableProperties - Property > > > > > > > delete.topic.enable > > > > > > > > > > > > is overridden to true > > > > > > > > > > > > > > > > > > > > > > > > I run this command: > > > > > > > > > > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand --zookeeper > > > > > > > > > > > > myzkconnect:12345/mynamespace --delete --topic > mytopic > > > > > > > > > > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection). > > > > > > > > > > > > log4j:WARN Please initialize the log4j system > properly. > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread). > > > > > > > > > > > > log4j:WARN See > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > > > for more info. > > > > > > > > > > > > log4j:WARN Please initialize the log4j system > properly. > > > > > > > > > > > > log4j:WARN See > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > > > for more info. > > > > > > > > > > > > Topic mytopic is marked for deletion. > > > > > > > > > > > > Note: This will have no impact if > delete.topic.enable is > > > > not > > > > > set > > > > > > > to > > > > > > > > > > true. > > > > > > > > > > > > > > > > > > > > > > > > I then do a —list which should at least show the > topic > > > > > marked for > > > > > > > > > > > deletion: > > > > > > > > > > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand --zookeeper > > > > > > > > > > > > myzkconnect:12345/mynamespace --list --topic mytopic > > > > > > > > > > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection). > > > > > > > > > > > > log4j:WARN No appenders could be found for logger > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread). > > > > > > > > > > > > log4j:WARN Please initialize the log4j system > properly. > > > > > > > > > > > > log4j:WARN Please initialize the log4j system > properly. > > > > > > > > > > > > log4j:WARN See > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > > > for more info. > > > > > > > > > > > > log4j:WARN See > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig > > > > > > > > > > > > for more info. > > > > > > > > > > > > mytopic > > > > > > > > > > > > > > > > > > > > > > > > Note, it doesn’t list it as ‘marked for deletion’. > > > > > Furthermore, > > > > > > > > > waiting > > > > > > > > > > > > multiple hours still doesn’t result in the topic > being > > > > > deleted. > > > > > > > > > > > > > > > > > > > > > > > > The topic has 1 partition, 2 replicas, and no data > > > stored. > > > > > > > > > > > > > > > > > > > > > > > > In zookeeper, the /admin/deleted_topics/ path is > empty. > > > > > > > > > > > > > > > > > > > > > > > > The zookeeper code looks pretty straightforward, but > for > > > > some > > > > > > > reason > > > > > > > > > is > > > > > > > > > > > not > > > > > > > > > > > > writing the deleted_topics path. We are running > > > zookeeper > > > > > 3.4.6. > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > Jason > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > >