Great, thanks!
On Tue, Mar 30, 2021 at 11:00 AM Till Rohrmann wrote:
> This is a good idea. I will add it to the section here [1].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/deployment/cli.html#terminating-a-job
>
> Cheers,
> Till
>
> On Tue, Mar 30, 2021 at 2:46 PM Vishal
This is a good idea. I will add it to the section here [1].
[1]
https://ci.apache.org/projects/flink/flink-docs-stable/deployment/cli.html#terminating-a-job
Cheers,
Till
On Tue, Mar 30, 2021 at 2:46 PM Vishal Santoshi
wrote:
> Got it. Is it possible to add this very important note to the
> doc
Got it. Is it possible to add this very important note to the
documentation. Our case is the former as in this is an infinite pipeline
and we were establishing the CiCD release process when non breaking
changes ( DAG compatible changes are made ) on a running pipe.
Regards
On Tue, Mar 30, 2021 at
Hi Vishal,
The difference between stop-with-savepoint and
stop-with-savepoint-with-drain is that the latter emits a max watermark
before taking the snapshot. The idea is to trigger all pending timers and
flush the content of some buffering operations like windowing.
Semantically, you should use th
More interested whether a StreamingFileSink without a drain
negatively affects it's exactly-once semantics , given that I state on SP
would have the offsets from kafka + the valid lengths of the part files at
SP. To be honest not sure whether the flushed buffers on sink are included
in the length
Is this a known issue. We do a stop + savepoint with drain. I see no back
pressure on our operators. It essentially takes a SP and then the SInk (
StreamingFileSink to S3 ) just stays in the RUNNING state.
Without drain i stop + savepoint works fine. I would imagine drain is
important ( flush the
