I think there could be correctness implications... the default cleanup.policy of delete would mean that topic entries past the retention policy might have been removed. If you scale up the application, new application instances won't be able to restore a complete table into its local state store. An operation like a join against that KTable would find no records where there should be record.
Mathieu On Wed, Feb 8, 2017 at 12:15 PM, Eno Thereska <eno.there...@gmail.com> wrote: > If you fail to set the policy to compact, there shouldn't be any > correctness implications, however your topics will grow larger than > necessary. > > Eno > > > On 8 Feb 2017, at 18:56, Jon Yeargers <jon.yearg...@cedexis.com> wrote: > > > > What are the ramifications of failing to do this? > > > > On Tue, Feb 7, 2017 at 9:16 PM, Matthias J. Sax <matth...@confluent.io> > > wrote: > > > >> Yes, that is correct. > >> > >> > >> -Matthias > >> > >> > >> On 2/7/17 6:39 PM, Mathieu Fenniak wrote: > >>> Hey kafka users, > >>> > >>> Is it correct that a Kafka topic that is used for a KTable should be > set > >> to > >>> cleanup.policy=compact? > >>> > >>> I've never noticed until today that the KStreamBuilder#table() > >>> documentation says: "However, no internal changelog topic is created > >> since > >>> the original input topic can be used for recovery"... [1], which seems > >> like > >>> it is only true if the topic is configured for compaction. Otherwise > the > >>> original input topic won't necessarily contain the data necessary for > >>> recovery of the state store. > >>> > >>> [1] > >>> https://github.com/apache/kafka/blob/e108a8b4ed4512b021f9326cf07951 > >> 7523c83060/streams/src/main/java/org/apache/kafka/streams/ > >> kstream/KStreamBuilder.java#L355 > >>> > >>> Thanks, > >>> > >>> Mathieu > >>> > >> > >> > >
