On 09.03.20 06:10, Rong Rong wrote:
- Is this feature (disabling checkpoint and restarting job from Kafka committed GROUP_OFFSET) not supported?
I believe the Flink community never put much (any?) effort into this because the Flink Kafka Consumer does its own offset handling. Starting from the committed offsets should work fine, though, the default startup mode is even StartupMode.GROUP_OFFSETS.
- How does Flink-Kafka actually handles auto-commit to coordinator given the fact that Flink ignores the coordinator assignments and uses self-assigning partitions instead?
I think we don't do anything for this case, the Kafka Consumer code will do the committing if 'enable.auto.commit' is set. I don't know how this will play with out code because we disable the automatic group handling.
Do you think letting Kafka do the auto committing is ever useful, if you have a Flink job that does checkpoints you will get the correct offset committing and you can start a job from the committed offsets. In what cases would you want to use the builtin Kafka offset committing?
Best, Aljoscha
