[
https://issues.apache.org/jira/browse/HDDS-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17270:
------------------------------
Status: Patch Available (was: Open)
> Terminate OM when snapshot purge fails after partially updating the snapshot
> cache and chain
> --------------------------------------------------------------------------------------------
>
> Key: HDDS-17270
> URL: https://issues.apache.org/jira/browse/HDDS-17270
> Project: Apache Ozone
> Issue Type: Bug
> Components: Snapshot
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> {{OMSnapshotPurgeRequest#validateAndUpdateCache}} purges the snapshots in a
> request one at a time. For each one it updates the snapshotInfoTable cache
> (deep clean flags and previous snapshot IDs of the next snapshots, plus a
> tombstone for the purged snapshot) and removes it from the in-memory
> {{SnapshotChainManager}}. If a later snapshot in the same request throws, the
> request returns an error response and none of these changes are rolled back,
> while the DB batch is skipped because the response is not OK (HDDS-11452).
> Most failures in that loop already terminate OM: a plain {{IOException}} maps
> to {{INTERNAL_ERROR}} or {{METADATA_ERROR}}, and
> {{OzoneManagerStateMachine#processResponse}} terminates OM on those, so the
> partial in-memory state is discarded. The exception is {{OMException}} with
> {{FILE_NOT_FOUND}} from {{SnapshotUtils#getNextSnapshot}}, thrown when the
> snapshot chain has a next snapshot that is missing from snapshotInfoTable.
> That status is not critical, so OM keeps running with a cache and chain that
> no longer match the DB. A later write of a successor SnapshotInfo can then
> persist a previous snapshot ID that skips a snapshot still present in the DB.
> This needs a purge request with more than one snapshot and a snapshot chain
> that already disagrees with snapshotInfoTable. With a single snapshot, both
> next snapshot lookups run before any change.
> h3. Fix
> Once the request has started changing the cache or snapshot chain, report an
> {{OMException}} as {{INTERNAL_ERROR}} so that OM terminates, consistent with
> how every other failure at that point is handled. On restart the chain is
> rebuilt from snapshotInfoTable.
> There is no protobuf, DB layout or upgrade change. The DB writes are the same
> as today in every case (a failed purge writes nothing), so OMs of different
> versions stay consistent. This does not make the request atomic; HDDS-11452
> stays open for staging or rolling back the changes so that OM can keep
> running.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
