Greetings!
I have problem with Kafka. I had a cluster of 3 brokers in version 0.8.1. I
have a very important topic with raw events, that had a config
retention.ms={365
days in ms} .
It all worked, fine, data was not being deleted.
But now I upgraded all brokers to 0.8.2 and suddenly brokers delete the
data! They don't respect retention.ms.
I have no other settings around retention set: global log.retention.hours
is set to default 168, log.retention.bytes is not set.
Some more info:
1. I tried looking into ZK config and it looks fine:
$ get /kafka081/config/topics/my_topic
{"version":1,"config":{"retention.ms":"31536000000"}}
cZxid = 0xa0006412c
ctime = Tue Mar 31 15:02:20 CEST 2015
mZxid = 0x1000016bb7
mtime = Fri Jun 26 22:28:40 CEST 2015
pZxid = 0xa0006412c
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 53
numChildren = 0
2. I tried to overwrite retention.ms once again for my topic. Didn't help.
3. I looked into logs of the broker and found that it indeed deletes the
data, but it doesn't print *why* (based on what rule) it deletes the data:
[2015-06-29 07:35:40,861] INFO Deleting segment 89226232 from log
my_topic-1. (kafka.log.Log)
[2015-06-29 07:35:40,993] INFO Deleting index
/var/lib/kafka/kafka-logs-1/my_topic-1/00000000000089226232.index.deleted (k
afka.log.OffsetIndex)
Please help me, I have no idea what to do about it. Any hint on at least
how to debug a problem would be great!
Cheers,
Krzysztof Zarzycki
