Hi Chen!
If I understand, you want to implement a custom way of triggering
checkpoints, based on messages in the input message queue (for example
based on Kafka events)? Basically to trigger a checkpoint when you have
received a special message through each Kafka partition?
Please let me know if
Hi there,
So far, checkpoint trigger is hardcoded in CheckpointCorrdinator which
triggered periodically and push control messages to task managers. It was
implemented orthogonal to business logics implemented in jobs.
Our scenario requires master pipeline flow control messages along with
events i
