Hi Kostas,
Thanks you for the clarification, also can you please point
how StreamingFileSink uses TwoPhaseCommit. Can you also point out the
implementing class for that?
Regards,
Taher Koitawala
GS Lab Pune
+91 8407979163
On Thu, Jan 10, 2019 at 3:10 PM Kostas Kloudas <kklou...@gmail.com> wrote:
> That is correct.
> The StreamingFileSink does not extend the TwoPhaseCommitSinkFunction
> because of some internal implementation details.
> But this, does not mean that it does not implement a two phase commit
> protocol (which is independent of the implementation).
>
> Cheers,
> Kostas
>
> On Thu, Jan 10, 2019 at 10:33 AM Taher Koitawala <
> taher.koitaw...@gslab.com> wrote:
>
>> StreamingFileSink extends RichSinkFunction and implements
>> CheckpointedFunction, CheckpointListener and ProcessingTimeCallback however
>> TwoPhaseCommitSinkFunction is never used by StreamingFileSink. Hence I had
>> a question if the sink uses the TwoPhaseCommit protocol or not.
>>
>> Regards,
>> Taher Koitawala
>>
>> On Thu 10 Jan, 2019, 2:40 PM Kostas Kloudas <kklou...@gmail.com wrote:
>>
>>> Hi Taher,
>>>
>>> The StreamingFileSink implements a version of TwoPhaseCommit. Can you
>>> elaborate a bit on what do you mean by " TwoPhaseCommit is not being
>>> used"?
>>>
>>> Cheers,
>>> Kostas
>>>
>>> On Thu, Jan 10, 2019 at 9:29 AM Taher Koitawala <
>>> taher.koitaw...@gslab.com> wrote:
>>>
>>>> Hi All,
>>>> As per my understanding and the API of StreamingFileSink,
>>>> TwoPhaseCommit is not being used. Can someone please confirm is that's
>>>> right? Also if StreamingFileSink does not support
>>>> TwoPhaseCommits what is the best way to implement this?
>>>>
>>>>
>>>> Regards,
>>>> Taher Koitawala
>>>> GS Lab Pune
>>>> +91 8407979163
>>>>
>>>
