Hi Maarten, Thanks for the KIP!
It looks good to me. It seems appropriate to stick close to the same API presented by Materialized. I did notice one mis-statement in the proposed Javadoc: > * Indicates that a changelog should be created for the suppressed KTable. Actually, the changelog is for the buffer only, not the KTable. This may be an important distinction for some configurations that folks might set. When the suppress operator gets data from upstream, it adds it to the buffer (and also sends that data to the changelog). When it emits records from the buffer, it also sends a tombstone to the changelog topic. Thus, data in the buffer is *much* shorter lived than data in other KTables, and the buffer changelog also may be much more compactable than KTable changelogs. I only bring this up because it might lead people to configure the changelog topic differently. One question that I have is whether we should place any restriction on the use of these methods. Since there is currently only an in-memory implementation of the suppression buffer, disabling the changelog is almost guaranteed to lead to data loss during rebalances or restarts. Then again, the same thing is true if you use an in-memory state store with changelog disabled. Thoughts? Thanks again for the KIP. It'll be great to get this in. -John On Tue, Apr 2, 2019 at 4:39 PM Maarten Duijn <maartendu...@msn.com> wrote: > Kafka Streams currently does not allow configuring the internal changelog > topic created by KTable.suppress. This KIP introduces a design for adding > topic configurations to the suppress API. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-446%3A+Add+changelog+topic+configuration+to+KTable+suppress > >
