Re: KafkaStreams - impact of retention on repartition topics

2019-08-26 Thread Murilo Tavares
Cool! Thank you Matthias! On Sun, 25 Aug 2019 at 15:11, Matthias J. Sax wrote: > You cannot delete arbitrary data, however, it's possible to send a > "truncate request" to brokers, to delete data before the retention time > is reached: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP

Re: KafkaStreams - impact of retention on repartition topics

2019-08-25 Thread Matthias J. Sax
You cannot delete arbitrary data, however, it's possible to send a "truncate request" to brokers, to delete data before the retention time is reached: https://cwiki.apache.org/confluence/display/KAFKA/KIP-107%3A+Add+deleteRecordsBefore%28%29+API+in+AdminClient There is `AdminClient#deleteRecords(

Re: KafkaStreams - impact of retention on repartition topics

2019-08-21 Thread Murilo Tavares
Thanks Matthias for the prompt response. Now just for curiosity, how does that work? I thought it was not possible to easily delete topic data... On Wed, Aug 21, 2019 at 4:51 PM Matthias J. Sax wrote: > No need to worry about this. > > Kafka Streams used "purge data" calls, to actively delete d

Re: KafkaStreams - impact of retention on repartition topics

2019-08-21 Thread Matthias J. Sax
No need to worry about this. Kafka Streams used "purge data" calls, to actively delete data from those topics after the records are processed. Hence, those topics won't grow unbounded but are "truncated" on a regular basis. -Matthias On 8/21/19 11:38 AM, Murilo Tavares wrote: > Hi > I have a co