devmadhuu opened a new pull request, #11377: URL: https://github.com/apache/ozone/pull/11377
## What changes were proposed in this pull request?
Keys now carry a storage policy end to end, so a key's data lands on the
storage tier its policy asks for.
Before this change, OM resolved
every key's placement to the cluster default. `OMKeyRequest.allocateBlock`
hardcoded it:
```
// TODO Use the actually passed `allowFallbackStoragePolicy` instead of
`true`
allocatedBlocks = ... .allocateBlock(...,
OzoneStoragePolicy.getDefaultPolicy(), true);
and BlockOutputStream hardcoded the datanode-side storage type, both left as
TODOs by HDDS-16388:
// TODO: Replica to the method parameter
blkIDBuilder.setStorageTypeID(StorageTypeUtils.getID(StorageType.DISK));
```
This patch resolves all three
TODOs. The chain is now:
1. `KeyArgs.storagePolicy / KeyInfo.storagePolicy` added to
`OmClientProtocol.proto` (fields 25 and 23).
2. `OmKeyArgs` and `OmKeyInfo` carry a StoragePolicy, round-tripped
through protobuf.
3. `OMKeyRequest.getStoragePolicy(bucketInfo, keyArgs)` resolves it —
key-level policy wins, then the bucket's, then the cluster default — and
`allocateBlock` passes the resolved policy and
`allowFallbackStoragePolicy` to SCM. Applied at all three call sites:
`OMKeyCreateRequest`, `OMKeyCreateRequestWithFSO`, `OMFileCreateRequest`,
`OMAllocateBlockRequest`.
4. The tier SCM chose reaches the datanode: `BlockOutputStreamEntryPool`
derives a `StorageType` from `OmKeyLocationInfo.getStorageTier()`, which
threads through `BlockOutputStreamEntry` →
`RatisBlockOutputStream` → `BlockOutputStream`, populating
`DatanodeBlockID.storageTypeID` and the `writeChunkAsync` storage type.
5. Client API: `OzoneBucket.createKey(key, size, replicationConfig,
metadata, tags, storagePolicy)`, and the policy is readable back via
`OzoneKey.getStoragePolicy()`.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16665
## How was this patch tested?
New integration test `TestOzoneStoragePolicy` (6 tests): explicit key policy
honoured, bucket inheritance and key-level override, cluster-default fallback,
policy visible to the client on read, FSO bucket path, and empty-key policy
recording.
Its datanode assertion checks physical placement, not just metadata:
```
assertEquals(expectedStorageType, containerData.getStorageType());
assertEquals(expectedStorageType,
containerData.getVolume().getStorageType());
assertEquals(expectedStorageType, StorageTypeUtils
.getStorageTypeFromID(blockData.getBlockID().getStorageTypeID()));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
