Hi Tony,
the "log.retention.hours" property is only for topic retention period, this
does not affect the topic's deletion.
Your data is correctly deleted but, the only way to remove a topic is to use
"./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic testTopic"
After that you topic is "marked" for deletion but not "really" deleted. In
order to do that, you must set "delete.topic.enable = true" into your
server.properties
your topic had been created when delete.topic.enable=true had not been set. You
must set that in config, restarted kafka to apply the new config. Deleted the
topic and saw the "marked for deletion". You then restarted kafka again. After
5 minutes though your topic had been deleted. If the topic has higher numbers
of partitions it to take longer ...
best regarde,
Adrien (if you prefer to talk in french, don't hesitate)
________________________________
De : Tony DOIN <tony.d...@thales-services.fr>
Envoyé : mercredi 4 avril 2018 07:53:16
À : users@kafka.apache.org
Objet : Kafka topic retention
Hi,
I'm working with KAFKA_2.11-1.0.0.
I'm using the following
configuration in server.properties for LOG RETENTION POLICY:
*
log.retention.hours=168
* log.segment.bytes=1073741824
*
log.retention.check.interval.ms=300000
I'm trying to delete the kafka
topic "testTopic", by overriding the log.retention.hours properties key
using the following command:
* kafka-topics.sh --alter --zookeeper
zookeeper --topic testTopic--config retention.ms=60000
Then, i can see
the new configuration is applied for the topic 'testTopic" using the
following command:
* kafka-topics.sh --describe --zookeeper zookeeper
--topic testTopic
* Topic:testTopic PartitionCount:1
ReplicationFactor:1 Configs:retention.ms=60000
Topic: testTopic
Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Once the
"log.retention.check.interval.ms" is applied by kafka (by default 5min),
my data (in kafka-logs/testTopic-0) are correctly deleted until the last
index of the topic.
But folder kafka-logs/testTopic-0 in the file
system and the topic "testTopic" in kafka UI are still present, whereas
there is no data available.
Why the folder kafka-logs is not deleted?
Is it the normal work using this configuration?
What is the best way
to delete kafka data for a topic, and finally delete the topic (to no
longer appear in kafka UI and in the file system) ?
Thanks,
