>
> Eyal Peer
>
> From: Dawid Wysakowicz
> Sent: Friday, April 24, 2020 4:20 PM
> To: Eyal Pe'er ; user
> Subject: Re: Fault tolerance in Flink file Sink
>
>
>
> Hi Eyal,
>
> First of all I would say a local filesystem is not a right choice for what
>
Fault tolerance in Flink file Sink
Hi Eyal,
First of all I would say a local filesystem is not a right choice for what you
are trying to achieve. I don't think you can achive a true exactly once policy
in this setup. Let me elaborate why.
Let me clarify a bit how the StreamingFileSink w
Forgot to cc Kostas
On 23/04/2020 12:11, Eyal Pe'er wrote:
>
> Hi all,
> I am using Flink streaming with Kafka consumer connector
> (FlinkKafkaConsumer) and file Sink (StreamingFileSink) in a cluster
> mode with exactly once policy.
>
> The file sink writes the files to the local disk.
>
> I’ve no
Hi Eyal,
First of all I would say a local filesystem is not a right choice for
what you are trying to achieve. I don't think you can achive a true
exactly once policy in this setup. Let me elaborate why.
Let me clarify a bit how the StreamingFileSink works. The interesting
bit is how it behaves
Hi all,
I am using Flink streaming with Kafka consumer connector (FlinkKafkaConsumer)
and file Sink (StreamingFileSink) in a cluster mode with exactly once policy.
The file sink writes the files to the local disk.
I've noticed that if a job fails and automatic restart is on, the task managers
loo
