Re: Kafka streams Processor life cycle behavior of close()

2016-10-08 Thread Srikanth
Tnx! Looks like fix is already in for 0.10.1.0 On Tue, Oct 4, 2016 at 6:18 PM, Guozhang Wang wrote: > Created https://issues.apache.org/jira/browse/KAFKA-4253 for this issue. > > > Guozhang > > On Tue, Oct 4, 2016 at 3:08 PM, Guozhang Wang wrote: > > > Hello Srikanth, > > > > We close the under

Re: Kafka streams Processor life cycle behavior of close()

2016-10-04 Thread Guozhang Wang
Created https://issues.apache.org/jira/browse/KAFKA-4253 for this issue. Guozhang On Tue, Oct 4, 2016 at 3:08 PM, Guozhang Wang wrote: > Hello Srikanth, > > We close the underlying clients before closing the state manager (hence > the states) because for example we need to make sure producer's

Re: Kafka streams Processor life cycle behavior of close()

2016-10-04 Thread Guozhang Wang
Hello Srikanth, We close the underlying clients before closing the state manager (hence the states) because for example we need to make sure producer's sent records have all been acked before the state manager records the changelog sent offsets as end offsets. This is kind of chicken-and-egg probl

Kafka streams Processor life cycle behavior of close()

2016-10-01 Thread Srikanth
Hello, I'm testing out a WriteToSinkProcessor() that batches records before writing it to a sink. The actual commit to sink happens in punctuate(). I also wanted to commit in close(). Idea here is, during a regular shutdown, we'll commit all records and ideally stop with an empty state. My commit(