Yumeng Zhang created FLINK-18706:
------------------------------------
Summary: Stop with savepoint cannot guarantee exactly-once for
kafka source
Key: FLINK-18706
URL: https://issues.apache.org/jira/browse/FLINK-18706
Project: Flink
Issue Type: Bug
Affects Versions: 1.11.1, 1.10.1
Reporter: Yumeng Zhang
When I run stop-with-savepoint command with my old job and submit a new job
with the previous sync-savepoint, I find sometimes my new job will consume a
few duplicate data. Here is my case. I have a data generation job with
parallelism 1, which will generate long number incrementally and send the data
to Kafka topicA which only has one partition. Then I have another consumer job
with parallelism 1, which reads data from topicA and does nothing processing,
just print these numbers to system out. For example, after doing
stop-with-savepoint, my consumer job has printed sequence
0,1,2,3...40,41,42,43. Then I start the consumer job again from that
sync-savepoint. It prints 41,42,43,44..., which means it has consumed some
duplicate data.
I think the reason is that we fail to guarantee the mutual exclusion between
canceling source task and sending data to downstream by checkpoint lock. It may
send some data to downstream first before sync-savepoint completed and then
cancel the task. Therefore, We need to keep the source operator running in the
synchronous savepoint mailbox loop for triggerCheckpoint method before
synchronous savepoint completed and keep checking running state before sending
data to downstream for Kafka connector.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
