[
https://issues.apache.org/jira/browse/HDFS-13021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341719#comment-16341719
]
Xiao Chen commented on HDFS-13021:
----------------------------------
Thanks for the comment [~szetszwo].
Since it's by design then maybe we can convert this into a doc jira. Didn't
find any mentions of this behavior in
[http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html,]
please comment if I'm missing it from some other docs.
{quote}what should HDFS do if the old and new storage policies are different?
{quote}
Agree this is tricky. Current behavior makes the difference impossible, and as
it's by design, changing it would be incompatible. As I can see from HDFS-6911
and related jiras, mover already has the logic to handle snapshots, so doc Jira
should be cool.
M2c on the problem itself is, storage policies on snapshots can be interpreted
as just part of the file metadata's snapshots. So old policies (on snapshot)
may not be accurately reflecting how the blocks are placed (even without
snapshot, it may also not be accurate, for example if it's changed and mover
not run yet), but they can be accurate on what the metadata was when the
snapshot was created, and immutable.
> Incorrect storage policy of snapshot file was returned by getStoragePolicy
> command
> ----------------------------------------------------------------------------------
>
> Key: HDFS-13021
> URL: https://issues.apache.org/jira/browse/HDFS-13021
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, snapshots
> Affects Versions: 3.1.0
> Reporter: LiXin Ge
> Assignee: LiXin Ge
> Priority: Major
> Attachments: HDFS-13021.001.patch
>
>
> Snapshots are supposed to be immutable and read only, so the file status
> which in a snapshot path shouldn't follow the original file's change.
> The StoragePolicy in snapshot situation acts like a bug now.
> -----------
> Reproduction：Operation on snapshottable dir {{/storagePolicy}}
> *before make snapshot:*
> {code:java}
> [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy
> PROVIDED
> Set storage policy PROVIDED on /storagePolicy
> [bin]# hadoop fs -put /home/file /storagePolicy/file_PROVIDED
> [bin]# hdfs storagepolicies -getStoragePolicy -path
> /storagePolicy/file_PROVIDED
> The storage policy of /storagePolicy/file_PROVIDED:
> BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[],
> replicationFallbacks=[ARCHIVE]}
> {code}
> *make snapshot and check:*
> {code:java}
> [bin]# hdfs dfs -createSnapshot /storagePolicy s3_PROVIDED
> Created snapshot /storagePolicy/.snapshot/s3_PROVIDED
> [bin]# hdfs storagepolicies -getStoragePolicy -path
> /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
> The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
> BlockStoragePolicy{PROVIDED:1, storageTypes=[PROVIDED, DISK],
> creationFallbacks=[PROVIDED, DISK], replicationFallbacks=[PROVIDED, DISK]}
> {code}
> *change the StroagePolicy and check again:*
> {code:java}
> [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy HOT
> Set storage policy HOT on /storagePolicy
> [bin]# hdfs storagepolicies -getStoragePolicy -path
> /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
> The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
> BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[],
> replicationFallbacks=[ARCHIVE]} ---- It shouldn't be HOT
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
