DataExchangeMode is Piped
If Two operators use Piped Mode to exchange the data , Failed partitions have
already send some data to the receiver before it failed.So Does Replaying all
the failed partitions cause some duplicate records ?
> Date: Thu, 9 Jul 2015 14:47:29 +0200
> Subject: Re: Does DataSet job also use Barriers to ensure "exactly once."?
> From: ktzou...@apache.org
> To: dev@flink.apache.org
>
> No, it doesn't; periodic snapshots are not needed in DataSet programs, as
> DataSets are of finite size and failed partitions can be replayed
> completely.
>
>
> On Thu, Jul 9, 2015 at 2:43 PM, 马国维 <maguo...@outlook.com> wrote:
>
> > hi, everyoneThe doc say Flink Streaming use "Barriers" to ensure
> > "exactly once."Does the DataSet job use the same mechanism to ensue
> > "exactly once" if a map task is failed?thanks
> >
