Hey Zach, That's correct. You probably want to look into the following topic-level configs:
cleanup.policy min.cleanable.dirty.ratio segment.ms segment.bytes (we usually use the default value) Also, here's some broker configs of interest that you might want to tweak (along with the settings we usually use... YMMV): log.cleaner.enable log.cleaner.io.buffer.load.factor (0.9) log.cleaner.io.buffer.size (524288) log.cleaner.backoff.ms (30000) log.cleaner.dedupe.buffer.size (524288000) log.cleaner.io.max.bytes.per.second (1000000000000.0) log.cleaner.delete.retention.ms (86400000) log.cleaner.min.cleanable.ratio (0.5) log.cleaner.threads (1) On a side note, sometimes it's nice to set min.cleanable.dirty.ratio to 0.01, then view the files on disk to make sure things are working. -Jon On Jan 26, 2015, at 2:42 PM, Zach Cox <zcox...@gmail.com> wrote: > Hi - in Samza 0.8.0 it seems that the Kafka topic created for a key-value > store changelog does not have compaction enabled, as described in this jira: > > https://issues.apache.org/jira/browse/SAMZA-226 > > If Samza creates this changelog topic, am I correct that we then later need > to run something like this to enable compaction (and smaller segment size)? > > bin/kafka-topics.sh --zookeeper whatever:2181/kafka --topic > "the-changelog-topic" --alter --config cleanup.policy=compact --config > segment.bytes=1000000 > > Thanks, > Zach
signature.asc
Description: Message signed with OpenPGP using GPGMail
