Hi Devs,

I am implementing a custom connector with the FLIP27 interface and I have a
question about how source events are handled with respect to
task/taskmanager failure. In my implementation, the enumerator sends source
events to readers so that readers can react to split changes detected by
the enumerator.

How are source events handled in the cases of task failure/task manager
failure? Are they completely lost or is there an internal mechanism to
re-apply source events upon reader/task/taskmanager restart?

I'm thinking I would need to keep track of processed source events in
enumerator state and resend them if they weren't processed before
reader/task/taskmanager failure. I'd like to avoid the extra work of this
mechanism and it sounds very similar to the addSplitsBack functionality, so
I'm looking for suggestions for what else I might be able to leverage.

Best,
Mason

Reply via email to