Re: ProcessorContext commit question

2017-02-09 Thread Eno Thereska
From: Eno Thereska [mailto:eno.there...@gmail.com] > Sent: 09 February 2017 16:14 > To: users@kafka.apache.org > Subject: Re: ProcessorContext commit question > > Hi Adrian, > > It's also done in the DSL, but at a different point, in Task.commit(), since > the

RE: ProcessorContext commit question

2017-02-09 Thread Adrian McCague
[mailto:eno.there...@gmail.com] Sent: 09 February 2017 16:14 To: users@kafka.apache.org Subject: Re: ProcessorContext commit question Hi Adrian, It's also done in the DSL, but at a different point, in Task.commit(), since the flow is slightly different. Yes, once data is stored in stores, the offsets shou

Re: ProcessorContext commit question

2017-02-09 Thread Eno Thereska
Hi Adrian, It's also done in the DSL, but at a different point, in Task.commit(), since the flow is slightly different. Yes, once data is stored in stores, the offsets should be committed, so in case of a crash the same offsets are not processed again. Thanks Eno > On 9 Feb 2017, at 16:06,

ProcessorContext commit question

2017-02-09 Thread Adrian McCague
Hi all, In processor and transformer implementations, what are the use cases for calling `context.commit()`? Examples imply it should be called when state store modifications are complete, Streams DSL implementations do not fall in line with the examples, ie KStreamAggregate. Thanks Adrian