Re: Kafka Streams and retention limits

2021-04-24 Thread Matthias J. Sax
The topics used by Kafka Streams to backup state stores, are called changelog topic, and they are configures with log compaction instead of retention, and thus, there is no size limit. Cf https://kafka.apache.org/documentation/#compaction -Matthias On 4/24/21 1:32 PM, Dan Bradley wrote: > I'm t

Kafka Streams and retention limits

2021-04-24 Thread Dan Bradley
I'm trying to understand the interactions between stateful processing in Kafka Streams and topic retention size restrictions. I'm fairly naive about both Kafka and Kafka Streams so I'm sure I have something wrong here, but my understanding is that, under stateful processing, local state in the