How would I know if we are seeing that issue? We are running 0.11.0.0 so we would not have this fix.
On Tue, Jan 9, 2018 at 11:07 AM Wim Van Leuven < wim.vanleu...@highestpoint.biz> wrote: > What minor version of Kafka are you running? Might you be impacted by > https://issues.apache.org/jira/browse/KAFKA-6030? > -w > > On Tue, 9 Jan 2018 at 19:02 Thunder Stumpges <thunder.stump...@gmail.com> > wrote: > > > Hello, I posted this on StackOverflow > > < > > > https://stackoverflow.com/questions/47948399/kafka-per-topic-retention-bytes-and-global-log-retention-bytes-not-working > > >also > > but haven't gotten any response. > > > > thanks in advance, > > Thunder > > __________ > > > > We are running a 6 node cluster of kafka 0.11.0. We have set a global as > > well as a per-topic retention in bytes, neither of which is being > applied. > > There are no errors that I can see in the logs, just nothing being > deleted > > (by size; the time retention does seem to be working) > > > > See relevant configs below: > > > > *./config/server.properties* : > > > > # global retention 75GB or 60 days, segment size 512MB > > log.retention.bytes=75000000000 > > log.retention.check.interval.ms=60000 > > > > log.retention.hours=1440 > > > > log.cleanup.policy=delete > > > > log.segment.bytes=536870912 > > > > *topic configuration (30GB):* > > > > [tstumpges@kafka-02 kafka]$ bin/kafka-topics.sh --zookeeper > > zk-01:2181/kafka --describe --topic stg_logtopic > > Topic:stg_logtopic PartitionCount:12 ReplicationFactor:3 > > Configs:retention.bytes=30000000000 > > Topic: stg_logtopic Partition: 0 Leader: 4 > > Replicas: 4,5,6 Isr: 4,5,6 > > Topic: stg_logtopic Partition: 1 Leader: 5 > > Replicas: 5,6,1 Isr: 5,1,6 > > ... > > > > And, disk usage showing 910GB usage for one partition! > > > > [tstumpges@kafka-02 kafka]$ sudo du -s -h /data1/kafka-data/* > > 82G /data1/kafka-data/stg_logother3-2 > > 155G /data1/kafka-data/stg_logother2-9 > > 169G /data1/kafka-data/stg_logother1-6 > > 910G /data1/kafka-data/stg_logtopic-4 > > > > I can see there are plenty of segment log files (512MB each) in the > > partition directory... what is going on?! > > > > Thanks in advance, Thunder > > >
