ub task is almost the same,
and there is no data skew. There are no exceptions in the program.
Best,
Ben
> On 6 Feb 2018, at 5:50 PM, xiaobin yan wrote:
>
> Hi,
>
> Thanks for your reply! See here
> :https://github.com/apache/flink/blob/master/flink-connectors/flink-connecto
thub.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointListener.java>
>
> 2018-02-06 4:14 GMT+01:00 xiaobin yan <mailto:yan.xiao.bin.m...@gmail.com>>:
> Hi ,
>
> You've got a point. I saw that method, but how can I m
. This
> would also be the right time to generate a _SUCCESS file.
> Have a look at the BucketingSink.notifyCheckpointComplete() method.
>
> Best, Fabian
>
>
>
>
> 2018-02-05 6:43 GMT+01:00 xiaobin yan <mailto:yan.xiao.bin.m...@gmail.com>>:
> Hi ,
>
>
tIndexOfThisSubtask()
> if (taskIndex == 0 && currentWatermark - lastSuccessWatermark > 1 hour) {
>Write _SUCCESS
>lastSuccessWatermark = currentWatermark round down to 1 hour
> }
> invoke(value)
> }
>
> Regards,
> Kien
> On 1/31/2018
Hi:
I think so too! But I have a question that when should I add this logic in
BucketingSink! And who does this logic, and ensures that the logic is executed
only once, not every parallel instance of the sink that executes this logic!
Best,
Ben
> On 31 Jan 2018, at 5:58 PM, Hung wrote:
>
> i
Hi:
How does BucketingSink generate a SUCCESS file when a directory is
finished， so that the downstream judge when the directory can be read.
Best
