Hello,
I have enabled checkpoint and I am using RollingSink to sink the data to HDFS
(2.7.x) from KafkaConsumer. To simulate failover/recovery, I stopped
TaskManager and the job gets rescheduled to other Taskmanager instance. During
this momemnt, the current "in-progress" gets closed and renamed to part-0-1
from _part-0-1_in-progress.
I was hoping to see the debug statement that I have added to "restoreState"
method but none of my debug statement gets printed. I am not sure if the
restoreState() method gets invoked during this scenario. Could you please help
me understand the flow during "failover" scenario?
P.S: Functionally the code appears to be working fine but I am trying to
understand the underlying implementation details. public void
restoreState(BucketState state)
Regards
Vijay
