Thanks to both !!. That's help me to understand the recovery process
El mié., 13 ene. 2016 a las 14:01, Stephan Ewen (<se...@apache.org>) escribió: > Thanks, Gordon, for the nice answer! > > One thing is important to add: Exactly-once refers to state maintained by > Flink. All side effects (changes made to the "outside" world), which > includes sinks, need in fact to be idempotent, or will only have "at-least > once" semantics. > > In practice, this works often very well, because results can be computes > with "exactly once" semantics in Flink and are then sent "one or more > times" to the outside world (for example a database). If that database > simply overwrites/replaces values for keys (think upsert operation), this > gives end-to-end exactly-once semantics. > > Greetings, > Stephan > > > On Wed, Jan 13, 2016 at 1:31 PM, Tzu-Li (Gordon) Tai <tzuli...@gmail.com> > wrote: > >> Hi Francis, >> >> A part of every complete snapshot is the record positions associated with >> the barrier that triggered the checkpointing of this snapshot. The >> snapshot >> is completed only when all the records within the checkpoint reaches the >> sink. When a topology fails, all the operators' state will fall back to >> the >> latest complete snapshot (incomplete snapshots will be ignored). The data >> source will also fall back to the position recorded with this snapshot, so >> even if there are repeatedly read data records after the restore, the >> restored operator's state are also clean of the records effect. This way, >> Flink guarantees exactly-once effects of each record on every operator's >> state. The user functions in operators need not to be implemented >> idempotent. >> >> Hope this helps answer your question! >> >> Cheers, >> Gordon >> >> >> >> -- >> View this message in context: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpoint-for-exact-once-proccessing-tp4261p4264.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> > >
