Re: Exactly-once publication behaviour

On 2/21/16 10:23 AM, Jay Kreps wrote: Hey Andrew, Yeah I think the current state is that we did several design and prototypes (both the transaction work and the idempotence design and the conditional write KIP), but none of these offshoots is really fully rationalized with the other ones. Slow p

RE: Exactly-once publication behaviour

Hi Jay, Thanks for the response. Happy to engage in the discussions once there's something concrete on the wiki. Andrew > Date: Sun, 21 Feb 2016 10:23:55 -0800 > Subject: Re: Exactly-once publication behaviour > From: j...@confluent.

Re: Exactly-once publication behaviour

ing on this subject? I'm not sure what > the appropriate forum is. How have you previously been sharing high-level > architectural plans for people to pick up and deliver? Since there's no KIP > I guess it's a bit premature to discuss on the KIP call. > > Thanks

RE: Exactly-once publication behaviour

;m not sure what the appropriate forum is. How have you previously been sharing high-level architectural plans for people to pick up and deliver? Since there's no KIP I guess it's a bit premature to discuss on the KIP call. Thanks Andrew > Subject: Re: Exactly-once publication behaviour >

Re: Exactly-once publication behaviour

Hi Andrew There are plans to add exactly once behaviour. This will likely be a little more than Idempotent producers with the motivation being to provide better delivery guarantees for Connect, Streams and Mirror Maker. B > On 19 Feb 2016, at 13:54, Andrew Schofield > wrote: > > When pu

Re: Exactly-once publication behaviour

Andrew, In SDC (https://github.com/streamsets/datacollector ) we do the kind of offset management you mention to achieve this type of behav