We've run into some unexpected behavior around this as well, though I forgot to send in a note when we found it so I'm fuzzy on the details at the moment. I'll chime back in if I can dig up exactly what we were doing, but I'd also welcome a ruling from someone with knowledge of the code. I seem to recall we tried this for a repartition topic and it didn't do quite what we expected.
On Fri, Aug 10, 2018 at 3:02 AM Patrik Kleindl <pklei...@gmail.com> wrote: > Hello > > In a discussion yesterday the question came up if an internal changelog > topic can be enabled for compaction and deletion. > > > https://stackoverflow.com/questions/50622369/kafka-streams-is-it-possible-to-have-compact-delete-policy-on-state-stores > and > https://issues.apache.org/jira/browse/KAFKA-4015 > say yes. > > https://kafka.apache.org/documentation/ > says yes for log.cleanup.policy on broker level > The default cleanup policy for segments beyond the retention window. A > comma separated list of valid policies. Valid policies are: "delete" and > "compact" > but no for cleanup.policy on topic level > A string that is either "delete" or "compact". > > My command line on 1.1 seems to agree with the last part: > ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics > --entity-name test --add-config log.cleanup.policy=compact,delete > requirement failed: Invalid entity config: all configs to be added must be > in the format "key=val". > ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics > --entity-name test --add-config cleanup.policy=compact,delete > requirement failed: Invalid entity config: all configs to be added must be > in the format "key=val". > ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics > --entity-name test --add-config cleanup.policy="compact,delete" > requirement failed: Invalid entity config: all configs to be added must be > in the format "key=val". > ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics > --entity-name test --add-config cleanup.policy='compact,delete' > requirement failed: Invalid entity config: all configs to be added must be > in the format "key=val". > > Is this missing for the client and/or the topic-level configuration > options? > > Does anyone know if/how this is supposed to work? > > Side-note: Our use-case (KTable with cleanup after some time) should be > covered with WindowKTables as far as I understand, but the documentation > for the cleanup.policy seems inconsistent. > > best regards > > Patrik >
