Antoine Philippot created FLINK-7883:
----------------------------------------
Summary: Stop fetching source before a cancel with savepoint
Key: FLINK-7883
URL: https://issues.apache.org/jira/browse/FLINK-7883
Project: Flink
Issue Type: Improvement
Components: DataStream API, Kafka Connector, State Backends,
Checkpointing
Affects Versions: 1.3.2, 1.4.0
Reporter: Antoine Philippot
For a cancel with savepoint command, the JobManager trigger the cancel call
once the savepoint is finished, but during the savepoint execution, kafka
source continue to poll new messages which will not be part of the savepoint
and will be replayed on the next application start.
A solution could be to stop fetching the source stream task before triggering
the savepoint.
I suggest to add an interface {{StoppableFetchingSourceFunction}} with a method
{{stopFetching}} that existant SourceFunction implementations could implement.
We can add a {{stopFetchingSource` property in {{CheckpointProperties}} and
{{CheckpointOptions}} class to pass the desired behaviour from
{{JobManager.handleMessage(CancelJobWithSavepoint)}} to
{{SourceStreamTask.triggerCheckpoint}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
