[
https://issues.apache.org/jira/browse/HDDS-17526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18136327#comment-18136327
]
Anurag Parvatikar commented on HDDS-17526:
------------------------------------------
Thanks [~deveshsingh] for the design doc. I have a few clarifying questions:
# With ZDU, old pipelines are not closed on finalization. After the Storage
Policy feature is finalized, these pipelines have supportedStorageTier=null, so
allocateBlock will not select them and they will stay open. Should they be
closed explicitly, or should a null tier be treated as the default tier?
# Pipeline and ContainerInfo both persist a StorageTier field. Should we gate
the persistence of these fields (and the datanode .container file's storage
type) behind a new HDDSVersion, so the on-disk data stays identical before
finalization?
# volumeChoosingPolicy picks a volume of the default configured StorageType
(DISK) when createContainer passes none. Should this be gated so that, before
finalization, any volume can be picked?
> ZDU Impact on Storage Policy Feature [HDDS-11233]
> -------------------------------------------------
>
> Key: HDDS-17526
> URL: https://issues.apache.org/jira/browse/HDDS-17526
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Datanode, Ozone Filesystem, Ozone Manager, Ozone
> Recon
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
> Attachments: Storage_Policy_ZDU_Design_with_Jira_Subtasks_v3.md
>
>
> The Storage Policy feature in HDDS-11233 is under development and started
> before the upstream zero-downtime upgrade (ZDU) framework was merged in
> master. The feature branch currently declares
> BUCKET_STORAGE_POLICY_SUPPORT(12) in the legacy OMLayoutFeature enum, and
> some OM request validators use that legacy value. After incorporating
> upstream ZDU, migrate this feature to the new component-version framework and
> establish explicit, testable compatibility boundaries for all components
> participating in policy metadata handling and physical data placement.
> OM must not persist or activate unsupported storage-policy metadata before
> the appropriate finalization. Clients and S3 Gateway must negotiate the
> capability of the receiving OM and preserve legacy behavior. SCM must
> validate policy-aware allocation, recovery and move capabilities
> independently of OM finalization; DataNodes must not silently ignore or
> misapply explicit storage-type requests. RATIS/EC repair, Container Balancer
> and Disk Balancer must maintain the documented strict/fallback placement
> contract. Recon and operational tooling must remain compatible with
> mixed-version metadata and report actionable failures.
> Deliver the implementation changes, version/field compatibility contract, and
> mixed-version unit/integration tests. Do not assume that ZDU finalization,
> Protobuf field acceptance, or RPC success proves actual physical placement
> compliance. Verify exact source paths and version numbers against the merged
> HDDS-11233 working tree.
> *Acceptance criteria:*
> * Storage Policy is registered through the correct post-ZDU OM
> component-version sequence, with an explicit decision for any persisted
> legacy feature-branch layout 12 state.
> * Pre-finalization OM operations preserve legacy behavior and do not persist
> new incompatible policy state; explicit-policy request handling is agreed and
> tested.
> * Client/S3G and each server boundary use appropriate negotiated capability
> checks and authoritative server-side enforcement.
> * Strict and allowed-fallback semantics are consistent across allocation,
> RATIS/EC repair, balancing and actual DataNode volume selection.
> * Mixed-version upgrade, restart, replay, snapshot, HA failover and
> downgrade-before-finalization scenarios pass, or have explicitly documented
> unsupported states.
> * Recon and operational signals remain usable throughout rollout;
> implementation and tests are linked to the corresponding tickets.
> cc: [~umamaheswararao] [~kkasawa] [~anuragparvatikar]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
