I think, at command line tool level, you need to use --add-config log.cleanup.policy=[compact,delete]
ie, you need to add square bracket to mark the config as a list.
This is different to Java code for which you would use
props.put("log.cleanup.policy", "compact,delete");
The config should be available at broker default as well as topic level
configuration. Seems there is a glitch in the docs. Please file a minor
PR or open a ticket so we can address this. Thx.
-Matthias
On 8/12/18 6:19 AM, Matt Farmer wrote:
> We've run into some unexpected behavior around this as well, though I
> forgot to send in a note when we found it so I'm fuzzy on the details at
> the moment. I'll chime back in if I can dig up exactly what we were doing,
> but I'd also welcome a ruling from someone with knowledge of the code. I
> seem to recall we tried this for a repartition topic and it didn't do quite
> what we expected.
>
> On Fri, Aug 10, 2018 at 3:02 AM Patrik Kleindl <pklei...@gmail.com> wrote:
>
>> Hello
>>
>> In a discussion yesterday the question came up if an internal changelog
>> topic can be enabled for compaction and deletion.
>>
>>
>> https://stackoverflow.com/questions/50622369/kafka-streams-is-it-possible-to-have-compact-delete-policy-on-state-stores
>> and
>> https://issues.apache.org/jira/browse/KAFKA-4015
>> say yes.
>>
>> https://kafka.apache.org/documentation/
>> says yes for log.cleanup.policy on broker level
>> The default cleanup policy for segments beyond the retention window. A
>> comma separated list of valid policies. Valid policies are: "delete" and
>> "compact"
>> but no for cleanup.policy on topic level
>> A string that is either "delete" or "compact".
>>
>> My command line on 1.1 seems to agree with the last part:
>> ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics
>> --entity-name test --add-config log.cleanup.policy=compact,delete
>> requirement failed: Invalid entity config: all configs to be added must be
>> in the format "key=val".
>> ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics
>> --entity-name test --add-config cleanup.policy=compact,delete
>> requirement failed: Invalid entity config: all configs to be added must be
>> in the format "key=val".
>> ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics
>> --entity-name test --add-config cleanup.policy="compact,delete"
>> requirement failed: Invalid entity config: all configs to be added must be
>> in the format "key=val".
>> ./kafka-configs --zookeeper broker0:2181 --alter --entity-type topics
>> --entity-name test --add-config cleanup.policy='compact,delete'
>> requirement failed: Invalid entity config: all configs to be added must be
>> in the format "key=val".
>>
>> Is this missing for the client and/or the topic-level configuration
>> options?
>>
>> Does anyone know if/how this is supposed to work?
>>
>> Side-note: Our use-case (KTable with cleanup after some time) should be
>> covered with WindowKTables as far as I understand, but the documentation
>> for the cleanup.policy seems inconsistent.
>>
>> best regards
>>
>> Patrik
>>
>
signature.asc
Description: OpenPGP digital signature
