devmadhuu commented on code in PR #11053: URL: https://github.com/apache/ozone/pull/11053#discussion_r4291021084
########## hadoop-hdds/docs/content/design/s3-object-lock.md: ########## @@ -0,0 +1,589 @@ +--- +title: S3 Object Lock +summary: Design to support S3 object lock. +date: 2026-08-18 +jira: HDDS-15945 +status: draft +author: Chung En Lee +--- +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# S3 Object Lock Design Doc + +## Summary + +This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger. +The primary objective is to provide data immutability and tamper-proof protection through the object locking feature. + +## Problem statement + +With growing demands for data security and compliance, ensuring that critical data stored in Ozone is +protected from accidental or malicious deletion and overwriting has become an essential system protection requirement. +To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature. + +Considering the current system architecture and access control strategies, +this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets. +Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration. +Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock), +Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid, +stable rollout of baseline tamper-proof protection. + +**Goal:** + +* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger. +* Support single-version objects only. + +## Non-Goal + +* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development). +* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase: + * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both. + * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID +* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase. + +## Differences from AWS + +* Single-Version Model: Locks apply to the current key rather than individual object versions. +* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException). +* Authorization: Ozone has no root account. + * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)). + * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)). + * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)). + * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)). + +## Technical Description + +### Terminology + +**Legal Hold** + +* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold). +* **Restricted Operations**: + * Put Object + * Delete Object + * Multipart Initial / Complete +* **Allowed Operations**: + * Get Object / Head Object + * Get / Put Object Tagging + * Get Legal Hold + * Put Legal Hold (Depends on permission) +* **Usage**: + * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept. + * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both, + and it stays locked until the legal hold is removed and the retention period has expired. + * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention. + +**Retention** + +* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown). +* Retention Modes: + * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires. + * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object. +* Event Hold / Event-based Retention: + * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure). + * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied. + * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys. + * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold; + the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released. + * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot. +* Restricted Operations: + * Put Object / Copy Object (Overwrites) + * Delete Object + * Multipart Upload (Initial / Complete) + * Shorten Retention Period (in Compliance Mode) +* Allowed Operations: + * Get Object / Head Object + * Get / Put Object Tagging + * Extend Retention Period + +#### Event Hold Trigger Workflow +For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions: + +```text +object created + | +event hold applied (indefinite WORM protection) + | +business event occurs + | +hold released + | +retention timer starts (RetainUntilDate is calculated) + | +WORM until timer expires +``` + +The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed +and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`). +When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock Review Comment: This is slightly contradicting what AWS provides. Consider a contract that must remain protected until closure. While it is still active, someone updates its minimum retention date. By this design, it say that says that `PutObjectRetention` on a key with an active event hold clears that hold. Protection could consequently end too early. AWS now provides an explicit `EventHold=OFF` field. Ozone should distinguish: - Updating retention while leaving the event hold active. - Explicitly releasing the event hold. - Setting or changing event-hold configuration. AWS also makes `RetainUntilDate` optional, so release should not necessarily require the caller to supply a date. The server can calculate it from the stored duration. [AWS PutObjectRetention (https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html)](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html) Could we provide an explicit event-hold release field? A normal `PutObjectRetention` update should preserve an active event hold unless release is requested. ########## hadoop-hdds/docs/content/design/s3-object-lock.md: ########## @@ -0,0 +1,589 @@ +--- +title: S3 Object Lock +summary: Design to support S3 object lock. +date: 2026-08-18 +jira: HDDS-15945 +status: draft +author: Chung En Lee +--- +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# S3 Object Lock Design Doc + +## Summary + +This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger. +The primary objective is to provide data immutability and tamper-proof protection through the object locking feature. + +## Problem statement + +With growing demands for data security and compliance, ensuring that critical data stored in Ozone is +protected from accidental or malicious deletion and overwriting has become an essential system protection requirement. +To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature. + +Considering the current system architecture and access control strategies, +this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets. +Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration. +Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock), +Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid, +stable rollout of baseline tamper-proof protection. + +**Goal:** + +* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger. +* Support single-version objects only. + +## Non-Goal + +* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development). +* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase: + * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both. + * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID +* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase. + +## Differences from AWS + +* Single-Version Model: Locks apply to the current key rather than individual object versions. +* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException). +* Authorization: Ozone has no root account. + * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)). + * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)). + * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)). + * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)). + +## Technical Description + +### Terminology + +**Legal Hold** + +* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold). +* **Restricted Operations**: + * Put Object + * Delete Object + * Multipart Initial / Complete +* **Allowed Operations**: + * Get Object / Head Object + * Get / Put Object Tagging + * Get Legal Hold + * Put Legal Hold (Depends on permission) +* **Usage**: + * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept. + * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both, + and it stays locked until the legal hold is removed and the retention period has expired. + * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention. + +**Retention** + +* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown). +* Retention Modes: + * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires. + * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object. +* Event Hold / Event-based Retention: + * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure). + * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied. + * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys. + * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold; + the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released. + * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot. +* Restricted Operations: + * Put Object / Copy Object (Overwrites) + * Delete Object + * Multipart Upload (Initial / Complete) + * Shorten Retention Period (in Compliance Mode) +* Allowed Operations: + * Get Object / Head Object + * Get / Put Object Tagging + * Extend Retention Period + +#### Event Hold Trigger Workflow +For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions: + +```text +object created + | +event hold applied (indefinite WORM protection) + | +business event occurs + | +hold released + | +retention timer starts (RetainUntilDate is calculated) + | +WORM until timer expires +``` + +The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed +and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`). +When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock +and sets `retentionDate` to the latest of the existing `retentionDate`, the requested `RetainUntilDate`, and the leader-stamped request time plus the event hold `rule`. +The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply. + +> _**Note**: +> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object. +> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects. + +### Table Changes + +**Bucket Table** + +Two new fields: objectLockEnabled & defaultRetention. + +```protobuf + message BucketInfo { + // ... existing fields + optional bool objectLockEnabled = 24 [default = false]; + optional BucketRetention defaultRetention = 25; + } + + message BucketRetention { + required RetentionMode retentionMode = 1; + optional Rule rule = 2; + optional EventHold eventHold = 3; + } + + message EventHold { + required bool enabled = 1; + required Rule rule = 2; + } + + message Rule { + required TimeUnit timeUnit = 1; + required uint64 duration = 2; + } + + enum TimeUnit { + DAYS = 1; + YEARS = 2; + } + + enum RetentionMode { + GOVERNANCE = 1; + COMPLIANCE = 2; + } +``` + + + +**Key Table** + +Two new fields: retention & legalHold. + +```protobuf +message KeyInfo { + // ... existing fields + optional KeyRetention retention = 23; + optional bool legalHold = 24 [default = false]; +} + +message KeyRetention { + required RetentionMode retentionMode = 1; + optional uint64 retentionDate = 2; // epoch milliseconds + optional EventHold eventHold = 3; +} +``` + + + +### Access Control + +Ozone Object Lock enforces a dual-gate protection mechanism combining **authorization (native ACLs or Ranger)** and **underlying OM WORM state validation**: + +* **Standard Data Operations (Put / Delete)**: Even if a principal has valid `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`). +* **Base Permissions**: Object Lock operations are checked against the existing `READ_ACL` / `WRITE_ACL` types for both native ACLs and Ranger, so no new ACL type is needed (see below). +* **Fine-Grained S3 Action Matching (Ranger)**: For requests through S3 Gateway, Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details. + +--- + +#### 1. Native ACL Mapping + +Changing Object Lock state is treated as a permission change, so it requires `WRITE_ACL` rather than `WRITE`. +A user who can only write or delete data cannot remove or shorten a lock. + +| Operation | Resource Level | Required Native ACL | +|:---------------------------------------|:---------------|:-----------------------------------| +| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` | +| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` | +| `GetObjectRetention` | Key | `READ_ACL` | +| `PutObjectRetention` | Key | `WRITE_ACL` | +| `GetObjectLegalHold` | Key | `READ_ACL` | +| `PutObjectLegalHold` | Key | `WRITE_ACL` | +| Bypass Governance retention | Key | Ozone administrator only | + +**Ozone administrator** means a user listed in `ozone.administrators` or a member of a group listed in `ozone.administrators.groups` +(if neither is set, only the user that starts the OM is an administrator). This is the same check used for other admin-only OM operations. +* The native authorizer already grants every ACL to Ozone administrators, so they also pass the `READ_ACL` / `WRITE_ACL` checks above. +* Bypass is limited to administrators because the native ACL model has no permission dedicated to it: any ACL type would also be grantable by bucket or key owners, + who could then remove the Governance protection from their own data. +* Read-only administrators (`ozone.readonly.administrators`) and S3 administrators (`ozone.s3.administrators`) cannot bypass Governance retention. +* The administrator role is cluster wide, so it cannot be scoped to a single volume or bucket. Deployments that need per-bucket bypass entitlement should use Ranger. +* The bypass does not apply to Compliance retention or a legal hold; a legal hold must be removed explicitly with `PutObjectLegalHold`, which an administrator can do because it only requires `WRITE_ACL`. + +--- + +#### 2. Ranger Authorization Modes + +With Ranger, every Object Lock request is authorized in one of two modes. +S3 actions are an S3 protocol concept, so only S3 Gateway sets them; OM does not add S3 actions to requests itself. +In both modes OM always checks the base access type (`READ_ACL` / `WRITE_ACL`), and the OM WORM state validation is the same. + +| Request source | `ozone.s3g.sts.http.enabled` | `enableActionMatcherInPoliciesCondition` | Mode | +|:--------------------------------|:-----------------------------|:-----------------------------------------|:------------------------------------------------------------------------| +| S3 Gateway | `true` | `true` | [Fine-Grained Mode](#3-prerequisite-for-s3-fine-grained-authorization) | +| S3 Gateway | `false` | `true` | [Base Permission Mode](#4-fallback-base-permission-mode); denied by policy items with an action condition (see below) | +| S3 Gateway | `false` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) | +| S3 Gateway | `true` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) (not recommended, see below) | +| Native Java API / CLI | any | any | [Base Permission Mode](#4-fallback-base-permission-mode); with the Ranger flag enabled, denied by policy items with an action condition | + +* **Fine-Grained Mode**: Ranger checks the S3 action (for example `PutObjectLegalHold`) and its base access type in the [mapping below](#6-action-to-ranger-access-type-mapping). +* **Base Permission Mode**: Ranger checks only the base access type. + +**Requests without an action when the Ranger flag is enabled**: + +When `enableActionMatcherInPoliciesCondition=true`, the result for a request without an S3 action (S3 Gateway with the STS flag disabled, or a native client) depends on the matching Ranger policy item: + +| Matching policy item | Request with an S3 action | Request without an S3 action | +|:----------------------------------------------|:---------------------------------------------------|:-----------------------------------------------------| +| Has an action condition (e.g. `PutObjectLegalHold`) | Granted only if the action is listed and the access type matches | **Denied**: the action condition cannot be satisfied | +| Has no action condition | Granted if the access type matches | Granted if the access type matches (Base Permission Mode) | + +* If the principal is granted Object Lock access only through policy items with an action condition, every Object Lock request without an action is denied, + including the requests that the same principal could send through S3 Gateway with both flags enabled. +* If the principal is granted the access type through a policy item without an action condition, the request falls back to Base Permission Mode. +* When `enableActionMatcherInPoliciesCondition=false`, action conditions are not evaluated at all, so every request is in Base Permission Mode. + +--- + +#### 3. Prerequisite for S3 Fine-Grained Authorization + +> _**Important for S3 users**_: AWS-style action-level authorization for Object Lock (separate permissions for `PutObjectRetention`, +> `PutObjectLegalHold`, `PutBucketObjectLockConfiguration` and `BypassGovernanceRetention`) is only available when **both** flags below are enabled. +> Both are `false` by default, so a cluster with default settings authorizes S3 Object Lock requests in [Base Permission Mode](#4-fallback-base-permission-mode). + +```properties +# S3 Gateway and OM: extract the S3 action and forward it to the authorizer +ozone.s3g.sts.http.enabled=true +# Ranger Ozone service definition: evaluate the S3 action in policy conditions +ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true +``` + +**Why both flags are required**: +* `ozone.s3g.sts.http.enabled` decides whether the S3 action reaches Ranger at all. Today S3 Gateway sets `S3Authentication.s3Action` and OM copies it into the `RequestContext` only when this flag is `true` + (`EndpointBase.applyS3Action`, `OmMetadataReader.maybeAddToContextFromThreadLocal`). The action was originally only needed for STS, which is why it is tied to this flag. +* `enableActionMatcherInPoliciesCondition` decides whether Ranger uses the action. When it is `false`, Ranger's `RangerActionMatcher` ignores the action and only checks the access type. +* If only the Ranger flag is enabled, S3 Gateway sends no action. Requests are denied by policy items with an action condition, and fall back to Base Permission Mode only through policy items without one + (see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)). S3 users granted Object Lock permissions by action would therefore be denied, so this combination is not recommended either. +* If only the STS flag is enabled, Ranger ignores the action, so requests also fall back to Base Permission Mode. + This combination is not recommended: STS session policies restrict temporary credentials by S3 action (see [AWS STS Design for Ozone S3](ozone-sts.md)), + so without the Ranger flag those restrictions would not be enforced. This is also why the STS flag is not enabled by default. + +**What S3 users get in Fine-Grained Mode**: +* Object Lock permissions can be granted separately, as in AWS IAM. For example, a legal team can be granted `PutObjectLegalHold` without `PutObjectRetention`, + and `BypassGovernanceRetention` can be granted to a smaller group than the one that sets retention. +* STS session policies (AssumeRole) can include or exclude Object Lock actions, and `IamSessionPolicyResolver` restricts the temporary credential accordingly. +* Each action still requires its base access type, so a wrong action mapping in S3 Gateway cannot reduce the check below `READ_ACL` / `WRITE_ACL`. + +**Operator guidance**: +* Clusters that expose Object Lock to S3 users with different roles (for example, data writers, compliance officers and administrators) should enable both flags before enabling Object Lock on any bucket. +* The STS flag only affects S3 Gateway requests. Native clients never carry an action, so they are denied by policy items with an action condition and are granted only through policy items without one (see [below](#4-fallback-base-permission-mode)). +* The user documentation of this feature will describe both flags and the difference between the two modes. + +--- + +#### 4. Fallback: Base Permission Mode + +Base Permission Mode is used for every request without an S3 action: +requests from the native Java API or CLI, and S3 Gateway requests when either flag in the [previous section](#3-prerequisite-for-s3-fine-grained-authorization) is disabled. +OM does not set an S3 action for these requests, so that the S3 action mapping stays in `S3GActionIamMapper` only. +When the Ranger flag is enabled, these requests are granted only through policy items without an action condition +(see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)). + +**Checks**: + +| Operation | Resource Level | Required Ranger Access Type | +|:---------------------------------------|:---------------|:----------------------------| +| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` | +| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` | +| `GetObjectRetention` | Key | `READ_ACL` | +| `PutObjectRetention` | Key | `WRITE_ACL` | +| `GetObjectLegalHold` | Key | `READ_ACL` | +| `PutObjectLegalHold` | Key | `WRITE_ACL` | +| Bypass Governance retention | Key | `WRITE_ACL`, `WRITE` and `DELETE` | + +**Why `READ_ACL` / `WRITE_ACL` instead of `READ` / `WRITE`**: +* Without an S3 action, a `WRITE` check could not tell `PutObject` from `PutObjectLegalHold`, so any user who can write a key could also remove its legal hold or shorten its retention. +* Changing a lock is treated as a permission change, so it uses the same access type as changing an ACL. Users with only `READ`, `WRITE`, `CREATE` or `DELETE` on a key cannot read or change its Object Lock settings. +* This matches the [Native ACL Mapping](#1-native-acl-mapping), so the same operation needs the same access type with native ACLs and with Ranger. + +**Limitations compared with Fine-Grained Mode**: +* All Object Lock changes on a resource need the same `WRITE_ACL`, so they cannot be granted separately. A principal that can set a legal hold can also set retention on the same key, and can bypass Governance retention if it also has `WRITE` and `DELETE`. +* Governance retention therefore only protects keys from principals without `WRITE_ACL`, `WRITE` and `DELETE`. Deployments that need a separate bypass entitlement should use Fine-Grained Mode. +* `WRITE_ACL` is also the access type for changing ACLs, so granting Object Lock management also grants that. +* Unlike native ACLs, where bypass is limited to Ozone administrators, Ranger has no Ozone administrator role, so bypass is granted through a policy with `WRITE_ACL`, `WRITE` and `DELETE`. +* Compliance retention is not affected: it cannot be removed in either mode. + +**Native clients when both flags are enabled**: +* Native clients carry no S3 action. A policy item with an action condition cannot be satisfied by a request without an action, so it does not grant the request. +* A principal whose `WRITE_ACL` is limited to `PutObjectLegalHold` through S3 Gateway is therefore denied all Object Lock operations through a native client; + it cannot use a native client to get around the action-level restrictions. +* To let a principal manage Object Lock through a native client, grant the access type through a policy item without an action condition. + That item applies to S3 Gateway requests as well, so the principal gets every Object Lock operation allowed by the access type, without action-level separation. + +--- + +#### 5. Ranger & S3 Gateway Component Updates + +1. **UI & Metadata Registration**: + * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger: + Register the 7 new Object Lock actions along with their prerequisite primitive permissions (`READ_ACL`, `WRITE_ACL`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items. + +2. **S3 Gateway & STS Mapping**: + * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string. + * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings, mapped to the same `READ_ACL` / `WRITE_ACL` base access types. + +--- + +#### 6. Action to Ranger Access Type Mapping + +In Fine-Grained Mode, following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy: + +| S3 Action | Resource Level | Required Ranger Access Type | Description | +|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------| +| **`GetBucketObjectLockConfiguration`** | Bucket | `READ_ACL` | Retrieve default Object Lock settings on a bucket. | +| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE_ACL` | Configure default retention mode and period on a bucket. | +| **`GetObjectRetention`** | Key | `READ_ACL` | Read the retention mode and Retain-Until date of an object version. | +| **`PutObjectRetention`** | Key | `WRITE_ACL` | Set or extend object retention mode and duration. | +| **`GetObjectLegalHold`** | Key | `READ_ACL` | Query the current Legal Hold status (`ON` or `OFF`). | +| **`PutObjectLegalHold`** | Key | `WRITE_ACL` | Toggle the Legal Hold state (`ON` or `OFF`). | +| **`BypassGovernanceRetention`** | Key | `WRITE_ACL`, `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. | + +--- + +#### 7. Retention Modes & Governance Bypass Enforcement + +* **Compliance Mode**: + * The strictest WORM protection tier. + * Once applied, the lock is immutable through the Ozone API. **No role, administrator, native ACL or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires. The only way to change it early is the offline `ozone repair` command described below. +* **Governance Mode & `BypassGovernanceRetention`**: + * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel. + * **Dual-Check Enforcement**: + 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument. + It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`). + Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag. + 2. **Bypass Entitlement**: With native ACLs, the principal must be an Ozone administrator (`ozone.administrators`). + With Ranger, the principal needs `WRITE_ACL`, `WRITE` and `DELETE` on the target key, and in Fine-Grained Mode also the **`BypassGovernanceRetention`** action. + 3. If either check fails, the request is aborted with `403 Access Denied`. + + +> _**Warning**: Permanent Data Immutability in Compliance Mode_ +> +> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account. +> Once applied, a Compliance lock cannot be overridden or deleted through the Ozone API by any admin, native ACL or Ranger policy until the retention period expires. +> It can only be changed offline with `ozone repair` (see below). + +### New Ozone APIs + +Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase. + +**OzoneBucket** + +```java +public void setRetentionConfig(RetentionArgs retentionArgs); +public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention); +public void setLegalHold(String keyName, boolean hold); +``` + +### Supported S3 APIs + +To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs. +In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read. + +**Bucket-Level APIs:** +* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header. +* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket. +* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets. +* Bucket links: S3 buckets are often links (for example `/s3v/bucket1` → `/vol1/bucket1`). Unlike `OMBucketSetPropertyRequest`, which rejects property changes on links, + `PutBucketObjectLockConfiguration` follows the link and stores the configuration on the source bucket, and `GetBucketObjectLockConfiguration` reads it from there. + Authorization is checked on both the link and the source bucket, like other requests that resolve links. Key operations already resolve links, so WORM checks always use the source bucket's configuration. + +**Object-Level APIs:** Review Comment: Below APIs doing and implicitly assuming permissions like updating retention date along with upload. Do we have a clear permission matrix ? For example: a client application writing data should not gain permission to choose a shorter retention period merely by adding a retention header to an upload. Equally, permission to manage retention should not grant permission to upload data. [AWS permission requirements (https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
