Re: Committing Kafka Transactions during Savepoint

Hi Scott, Some more clarifications: Doing a stop-with-savepoint will suspend the checkpoint coordinator, meaning that no new checkpoints will happen between taking the savepoint and shutting down the job. This means you will be save from duplicates if you only use savepoints for this. Regardi

Re: Committing Kafka Transactions during Savepoint

Hi Scott, For EXACTLY_ONCE in sink end with Kafka 0.11+ producer, The answer is YES. There is a official documentation you can have a good knowledge of this topic[1]. [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/connectors/kafka.html#kafka-011 Thanks, vino. 2018-07-27

Re: Committing Kafka Transactions during Savepoint

Thank you, Aljoscha! Are Kafka transactions committed when a running job has been instructed to cancel with a savepoint (e.g. `flink cancel -s `)? This is my primary use for savepoints. I would expect that when a new job is submitted with the savepoint, as in the case of an application upgrade,

Re: Committing Kafka Transactions during Savepoint

Hi, this has been in the back of my head for a while now. I finally created a Jira issue: https://issues.apache.org/jira/browse/FLINK-9983 In there, I also outline a better fix that will take a bit longer to implement. Best, Aljoscha > On 26.

Committing Kafka Transactions during Savepoint

I recently began using the exactly-once processing semantic with the Kafka 0.11 producer in Flink 1.4.2. It's been working great! Are Kafka transactions committed when creating a Flink savepoint? How does this affect the recovery behavior in Flink if, before the completion of the next checkpoint,