Matthias Pohl created FLINK-26175:
-------------------------------------
Summary: Make local cleanup being called in global cleanup
situations
Key: FLINK-26175
URL: https://issues.apache.org/jira/browse/FLINK-26175
Project: Flink
Issue Type: Technical Debt
Components: Runtime / Coordination
Affects Versions: 1.16.0
Reporter: Matthias Pohl
We introduce new interfaces for local cleanup and global cleanup in
FLINK-25432. The initial plan was to call them independently of each other. The
motivation behind it was to allow independent cleanup logic (e.g. in the
`DefaultJobGraphStore` for releasing and deleting the data from the
`StateHandleStore`).
This can be improved by adding an order to cleanup and calling the local
cleanup on each resource before calling the corresponding global cleanup in the
Dispatcher.
This works under the assumption that it's ok to keep the resource in an
inconsistent state during cleanup (i.e. it's ok for the resource to be in a
state were the local artifacts are already cleaned but the global artifacts are
not). That holds true because of how the {{Dispatcher}} interacts with the
{{JobResultStore}}. It functions as a transaction manager for the cleanup here.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
