Re: Transactional markers are not deleted from log segments when policy is compact

2019-06-03 Thread Pranavi Chandramohan
t; > > https://github.com/apache/kafka/pull/6715/files#diff-d7330411812d23e8a34889bee42fedfeR622 > > > Cheers! > -- > Jonathan > > > > > On Mon, Jun 3, 2019 at 7:02 AM Pranavi Chandramohan < > prana...@campaignmonitor.com> wrote: > > > Hi Jonathan, &

Re: Transactional markers are not deleted from log segments when policy is compact

2019-06-02 Thread Pranavi Chandramohan
expired. Our expectation is that the markers of inactive ProducerIds should be deleted. Does this hold true? Thanks, Pranavic On Sat, Jun 1, 2019 at 8:45 AM Pranavi Chandramohan < prana...@campaignmonitor.com> wrote: > Thanks Jonathan! That should help. > > On Fri, 31 May 2019, 6

Re: Transactional markers are not deleted from log segments when policy is compact

2019-05-31 Thread Pranavi Chandramohan
t helps. > > > Cheers! > -- > Jonathan > > > > > On Fri, May 31, 2019 at 8:59 AM Pranavi Chandramohan < > prana...@campaignmonitor.com> wrote: > > > Hi all, > > > > We use Kafka version 2.11-1.1.1. We produce and consume transactional > > messa

Transactional markers are not deleted from log segments when policy is compact

2019-05-31 Thread Pranavi Chandramohan
Hi all, We use Kafka version 2.11-1.1.1. We produce and consume transactional messages and recently we noticed that 2 partitions of the __consumer_offset topic have very high disk usage (256GB) When we looked at the log segments for these 2 partitions, there were files that were 6 months old. By d