Re: Exactly once processing

2016-04-20 Thread Sabarish Sasidharan
ing where you have persistent state stores, you > > need > > > to > > > > maintain the checkpoint which includes the committed offsets as well > as > > > the > > > > store flushed in sync, but right not these two operations are not > done > > > &

Re: Exactly once processing

2016-04-18 Thread Yi Pan
itted offsets while some of > > them > > > have already updated the stores. > > > > > > Guozhang > > > > > > > > > On Thu, Apr 14, 2016 at 11:56 PM, Sasidharan, Sabarish < > > > sabarish.sasidha...@harman.com>

Re: Exactly once processing

2016-04-15 Thread Robert Crim
hem > > have already updated the stores. > > > > Guozhang > > > > > > On Thu, Apr 14, 2016 at 11:56 PM, Sasidharan, Sabarish < > > sabarish.sasidha...@harman.com> wrote: > > > > > Hi > > > > > > To achieve exactly once pro

Re: Exactly once processing

2016-04-15 Thread Sabarish Sasidharan
2016 at 11:56 PM, Sasidharan, Sabarish < > sabarish.sasidha...@harman.com> wrote: > > > Hi > > > > To achieve exactly once processing for my aggregates, wouldn’t it be > > enough if I maintain the latest offset processed for the aggregate and > > check against th

Re: Exactly once processing

2016-04-15 Thread Guozhang Wang
duplicates where you consume from the committed offsets while some of them have already updated the stores. Guozhang On Thu, Apr 14, 2016 at 11:56 PM, Sasidharan, Sabarish < sabarish.sasidha...@harman.com> wrote: > Hi > > To achieve exactly once processing for my aggregates,

Exactly once processing

2016-04-15 Thread Sasidharan, Sabarish
Hi To achieve exactly once processing for my aggregates, wouldn’t it be enough if I maintain the latest offset processed for the aggregate and check against that offset when messages are replayed on recovery? Am I missing something here? Thanks Regards Sab

Exactly once processing

2016-04-15 Thread Sabarish Sasidharan
Hi To achieve exactly once processing for my aggregates, wouldn’t it be enough if I maintain the latest offset processed for the aggregate and check against that offset when messages are replayed on recovery? Am I missing something here? Thanks Regards Sab