ChenSammi commented on PR #10822: URL: https://github.com/apache/ozone/pull/10822#issuecomment-5406382033
> @ChenSammi > > > On decoupling versionId from the Ratis transaction index > > The goal here is to stop deriving the id from the replication log — versionId is user-visible and permanent, and the versionedKeyTable layout depends on it being ordered, which is the property most at risk if OM's execution model changes. > > First look was SCM's SequenceIdGenerator, since it solves the same problem. It doesn't fit: it lives in server-scm and is built on SCMHAManager / DBTransactionBuffer / @replicate, so reusing it would couple OM's write path to SCM across a module boundary. Its shape is still instructive though — a cheap local allocation plus a floor that guarantees monotonicity. > > So instead, looking at how object stores identify a version: a GCS generation is a microsecond timestamp, an Azure Blob version id is an ISO-8601 instant, and RocksDB's user-defined timestamps order versions by a key + timestamp suffix — the same encoding we already use in the dbKey. All three make the timestamp the sort key. > > Proposal: timestamp + max. > > The OM handling the request proposes a versionId in preExecute — wall clock in microseconds. preExecute's output is what goes to Ratis, so the id is fixed before replication. On apply, VersionIdAllocator settles it: max(proposedVersionId, currentVersionId + 1). A clock can't promise per-key ordering alone (same-microsecond writes, or a lagging clock after leader change), so the proposal is only a floor. The current version is already in hand, so this costs no read and stays a pure function of replicated state. > > That gives SCM's local-allocation-plus-floor structure without its batch machinery, since a versionId only has to increase within one key. > > Cost: one optional field on the commit / delete / complete-MPU requests. No new table, no extra I/O. Thanks @symious for the further investigation. The timestamp solution looks good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
