. That function would however still need the info which
>> partitions were updated/added.
>> And in my case, I was not really interested in the watermark (I sent it
>> downstream only for metric purposes) but want to tell impala after each
>> commit which partitions changed, regardless
ng a notifier sounds great, if it can be executed with parallelism 1 for
all sink tasks..
Best regards
Theo
Von: "Guowei Ma"
An: "Theo Diefenthal"
CC: "user" , "yungao gy"
Gesendet: Mittwoch, 13. Mai 2020 09:15:37
Betreff: Re: 回复：Re: Writing _SUC
quot; , "Jingsong Li" <
> jingsongl...@gmail.com>
> *CC: *"Peter Groesbeck" , "user" <
> user@flink.apache.org>
> *Gesendet: *Dienstag, 12. Mai 2020 10:36:59
> *Betreff: *回复：Re: Writing _SUCCESS Files (Streaming and Batch)
>
> Hi Peter,
>
mark (I sent it
> downstream only for metric purposes) but want to tell impala after each
> commit which partitions changed, regardless of the value from the watermark.
>
> Best regards
> Theo
>
> --
> *Von: *"Yun Gao"
> *An: *"Rober
for metric purposes) but want to tell impala after each commit
which partitions changed, regardless of the value from the watermark.
Best regards
Theo
Von: "Yun Gao"
An: "Robert Metzger" , "Jingsong Li"
CC: "Peter Groesbeck" , "user"
Hi Peter,
Sorry for missing the question and response later, I'm currently sworking
together with Jingsong on the issue to support "global committing" (like
writing _SUCCESS file or adding partitions to hive store) after buckets
terminated. In 1.11 we may first support watermark/time relate
