Hello
I understand that the compacted topic is meant to keep at least the latest
key value pair.
However, I am having an issue since it can happen that entry becomes old
and I need to remove it. It may also occur, that I am not able to send key
"null" pair. So I need another method to remove my hanging entries. My hope
was with the following configuration:
Topic:test.topic PartitionCount:30 ReplicationFactor:2
Configs:segment.bytes=1048576,min.cleanable.dirty.ratio=0.1,
delete.retention.ms=1800000,retention.ms=900000,segment.ms
=900000,cleanup.policy=compact,delete
Topic: test.topic Partition: 0 Leader: 0 Replicas: 1,0 Isr:
0
Topic: test.topic Partition: 1 Leader: 0 Replicas: 0,1 Isr:
0
Topic: test.topic Partition: 2 Leader: 0 Replicas: 1,0 Isr:
0
Topic: test.topic Partition: 3 Leader: 0 Replicas: 0,1 Isr:
0
Topic: test.topic Partition: 4 Leader: 0 Replicas: 1,0 Isr:
0
Topic: test.topic Partition: 5 Leader: 0 Replicas: 0,1 Isr:
0
Topic: test.topic Partition: 6 Leader: 0 Replicas: 1,0 Isr:
0
Topic: test.topic Partition: 7 Leader: 0 Replicas: 0,1 Isr:
0
Topic: test.topic Partition: 8 Leader: 0 Replicas: 1,0 Isr:
0
Topic: test.topic Partition: 9 Leader: 0 Replicas: 0,1 Isr:
0
Topic: test.topic Partition: 10 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 11 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 12 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 13 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 14 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 15 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 16 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 17 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 18 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 19 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 20 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 21 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 22 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 23 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 24 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 25 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 26 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 27 Leader: 0 Replicas: 0,1
Isr: 0
Topic: test.topic Partition: 28 Leader: 0 Replicas: 1,0
Isr: 0
Topic: test.topic Partition: 29 Leader: 0 Replicas: 0,1
Isr: 0
but it does not get cleaned. I stop writing one key value pair, but still
can see that key value pair after days.
Then I changed broker configuration on the fly:
# The minimum age of a log file to be eligible for deletion
#log.retention.hours=168
log.retention.minutes=16
# A size-based retention policy for logs. Segments are pruned from the log
as long as the remaining
# segments don't drop below log.retention.bytes.
#log.retention.bytes=1073741824
log.retention.bytes=10000
# The maximum size of a log segment file. When this size is reached a new
log segment will be created.
#log.segment.bytes=1073741824
log.segment.bytes=10000
So far I haven't been able to drop away the desired key value pair with any
of these configuration actions.
Am I trying something that is not possible or am I missing some
configuration options or should this work?
br.
Asko Alhoniemi
