Ok, thanks for the clarification. Really appreciate your help Kostas
On Thu 10 Jan, 2019, 6:19 PM Kostas Kloudas Hi Taher,
>
> Well, I would say there is no single class that implements it.
> In a nutshell, it is the StreamingFileSink that (through Buckets) tells
> the responsible Bucket what to
Hi Taher,
Well, I would say there is no single class that implements it.
In a nutshell, it is the StreamingFileSink that (through Buckets) tells the
responsible Bucket what to do at each step of the lifecycle of the Flink
operator
(mainly on element, on checkpoint, on checkpoint completed and on r
Hi Kostas,
Thanks you for the clarification, also can you please point
how StreamingFileSink uses TwoPhaseCommit. Can you also point out the
implementing class for that?
Regards,
Taher Koitawala
GS Lab Pune
+91 8407979163
On Thu, Jan 10, 2019 at 3:10 PM Kostas Kloudas wrote:
>
That is correct.
The StreamingFileSink does not extend the TwoPhaseCommitSinkFunction
because of some internal implementation details.
But this, does not mean that it does not implement a two phase commit
protocol (which is independent of the implementation).
Cheers,
Kostas
On Thu, Jan 10, 2019 a
StreamingFileSink extends RichSinkFunction and implements
CheckpointedFunction, CheckpointListener and ProcessingTimeCallback however
TwoPhaseCommitSinkFunction is never used by StreamingFileSink. Hence I had
a question if the sink uses the TwoPhaseCommit protocol or not.
Regards,
Taher Koitawala
Hi Taher,
The StreamingFileSink implements a version of TwoPhaseCommit. Can you
elaborate a bit on what do you mean by " TwoPhaseCommit is not being used"?
Cheers,
Kostas
On Thu, Jan 10, 2019 at 9:29 AM Taher Koitawala
wrote:
> Hi All,
> As per my understanding and the API of Streami
Hi All,
As per my understanding and the API of StreamingFileSink,
TwoPhaseCommit is not being used. Can someone please confirm is that's
right? Also if StreamingFileSink does not support
TwoPhaseCommits what is the best way to implement this?
Regards,
Taher Koitawala
GS Lab
