[
https://issues.apache.org/jira/browse/HDDS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janus Chow updated HDDS-16093:
------------------------------
Description:
This ticket includes the following tasks:
|*Sub-task*|*Scope*|*Acceptance*|
|T6.1 lifecycle proto + action wiring|noncurrentVersionExpiration on
LifecycleAction, expiredObjectDeleteMarker on LifecycleExpiration;
validation and the three S3 lifecycle endpoints accept them|rules round-trip
through Put / Get / DeleteBucketLifecycleConfiguration; invalid combinations
rejected|
|T6.2 versionedKeyTable scan in LifecycleActionTask|a second scan alongside the
existing keyTable scan, reusing its batching, saved scan state and metrics;
reclamation goes through a request of its own, since DeleteKeys addresses keys
by name and cannot name a version|noncurrent versions past NoncurrentDays
deleted with quota deducted; NewerNoncurrentVersions keeps N per key,
oldest-first; a pass is bounded by the versions it reads, not the ones it
selects, so a bucket whose versions all survive still yields between passes; it
stops only at a key boundary, so a key is never evaluated against half of its
own versions; versions permanently deleted or promoted since the pass selected
them are skipped|
|T6.3 Expiration inserts a marker|on a bucket that has ever been versioned,
Expiration.Days / .Date inserts a delete marker through the insertion the write
requests already share (T3.4) instead of deleting; a key whose current version
is already a marker is skipped|an expired current version is demoted to the
versionedKeyTable under a new marker rather than removed — still readable by
versionId, its blocks not reclaimed; markers never stack, so the version chain
does not grow for as long as the rule exists; the run reports markers inserted
rather than keys deleted with their bytes freed, since nothing was freed;
unversioned buckets unchanged|
|T6.4 ExpiredObjectDeleteMarker|removes a delete marker, and the key with it,
once nothing is left under it; the condition is "no version below the marker",
not the marker's own age; removal goes through the reclaim request, not
DeleteKeys, which on a versioned bucket would answer a delete by inserting
another marker|a key whose only remaining version is a marker disappears from
both tables, releasing namespace and nothing else; a key that still has
noncurrent versions is left alone, so promotion can never resurrect an object
the user deleted; the condition is re-checked under the bucket lock, where a
version written since the scan is still caught; a marker a write has superseded
is refused|
|T6.5 lifecycle service default and coupling|ozone.lifecycle.service.enabled
flips to true; OM refuses to enable versioning on a bucket while the service is
off, on both the create and the set-property path, checked in
preExecute|enabling versioning with the service disabled returns
NOT_SUPPORTED_OPERATION with an explanatory message on both paths; suspending
is not refused, since it can only reduce what has to be reclaimed; with the
service running, both paths behave as before|
was:
This ticket includes the following tasks:
|*Sub-task*|*Scope*|*Acceptance*|
|T6.1 lifecycle proto + action wiring|noncurrentVersionExpiration on
LifecycleAction, expiredObjectDeleteMarker on LifecycleExpiration;
validation and the three S3 lifecycle endpoints accept them|rules round-trip
through Put / Get / DeleteBucketLifecycleConfiguration; invalid combinations
rejected|
|T6.2 versionedKeyTable scan in LifecycleActionTask|a second scan alongside the
existing keyTable scan, reusing its batching, saved scan state and
metrics|noncurrent versions past NoncurrentDays deleted with quota deducted;
NewerNoncurrentVersions keeps N per key, oldest-first; the scan resumes from
saved state;
The scan resumes from saved state at a key boundary, so a key is never
evaluated against half of its own versions|
|T6.3 Expiration inserts a marker|on an ENABLED or SUSPENDED bucket,
Expiration.Days / .Date inserts a delete marker through the insertion the write
requests already share (T3.4) instead of deleting;
an unversioned bucket keeps today's actual delete|an expired current version is
demoted to the versionedKeyTable under a new marker rather than removed — still
readable by versionId, its blocks not reclaimed; a SUSPENDED bucket takes the
null slot with T5.1's semantics; unversioned buckets byte-for-byte unchanged|
|T6.4 ExpiredObjectDeleteMarker|removes a delete marker, and the key with it,
once nothing is left under it; the condition is "no version below the marker",
not the marker's own age|a key whose only remaining version is a marker
disappears from both tables;
a key that still has noncurrent versions is left alone, so promotion can never
resurrect an object the user deleted;
markers reachable by versionId until then|
|T6.5 maxVersions backstop and defaults|the per-bucket cap folded into the same
scan, oldest-first, markers counting toward it; an explicit
NewerNoncurrentVersions rule overrides it; ozone.lifecycle.service.enabled
flips to true and OM rejects PutBucketVersioning(Enabled) while the service is
off|a bucket with no lifecycle configuration converges to the cap; where both
apply the rule's N wins; boundaries (0 = unlimited, 1, cluster default)
covered; enabling versioning with the service disabled returns an explanatory
error|
> [Ozone versioning] [T6] Version-aware lifecycle
> -----------------------------------------------
>
> Key: HDDS-16093
> URL: https://issues.apache.org/jira/browse/HDDS-16093
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Labels: pull-request-available
>
> This ticket includes the following tasks:
> |*Sub-task*|*Scope*|*Acceptance*|
> |T6.1 lifecycle proto + action wiring|noncurrentVersionExpiration on
> LifecycleAction, expiredObjectDeleteMarker on LifecycleExpiration;
> validation and the three S3 lifecycle endpoints accept them|rules round-trip
> through Put / Get / DeleteBucketLifecycleConfiguration; invalid combinations
> rejected|
> |T6.2 versionedKeyTable scan in LifecycleActionTask|a second scan alongside
> the existing keyTable scan, reusing its batching, saved scan state and
> metrics; reclamation goes through a request of its own, since DeleteKeys
> addresses keys by name and cannot name a version|noncurrent versions past
> NoncurrentDays deleted with quota deducted; NewerNoncurrentVersions keeps N
> per key, oldest-first; a pass is bounded by the versions it reads, not the
> ones it selects, so a bucket whose versions all survive still yields between
> passes; it stops only at a key boundary, so a key is never evaluated against
> half of its own versions; versions permanently deleted or promoted since the
> pass selected them are skipped|
> |T6.3 Expiration inserts a marker|on a bucket that has ever been versioned,
> Expiration.Days / .Date inserts a delete marker through the insertion the
> write requests already share (T3.4) instead of deleting; a key whose current
> version is already a marker is skipped|an expired current version is demoted
> to the versionedKeyTable under a new marker rather than removed — still
> readable by versionId, its blocks not reclaimed; markers never stack, so the
> version chain does not grow for as long as the rule exists; the run reports
> markers inserted rather than keys deleted with their bytes freed, since
> nothing was freed; unversioned buckets unchanged|
> |T6.4 ExpiredObjectDeleteMarker|removes a delete marker, and the key with it,
> once nothing is left under it; the condition is "no version below the
> marker", not the marker's own age; removal goes through the reclaim request,
> not DeleteKeys, which on a versioned bucket would answer a delete by
> inserting another marker|a key whose only remaining version is a marker
> disappears from both tables, releasing namespace and nothing else; a key that
> still has noncurrent versions is left alone, so promotion can never resurrect
> an object the user deleted; the condition is re-checked under the bucket
> lock, where a version written since the scan is still caught; a marker a
> write has superseded is refused|
> |T6.5 lifecycle service default and coupling|ozone.lifecycle.service.enabled
> flips to true; OM refuses to enable versioning on a bucket while the service
> is off, on both the create and the set-property path, checked in
> preExecute|enabling versioning with the service disabled returns
> NOT_SUPPORTED_OPERATION with an explanatory message on both paths; suspending
> is not refused, since it can only reduce what has to be reclaimed; with the
> service running, both paths behave as before|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
