Hi,
I am working on implementing a custom placement policy. So I created a
class that extends BlockPlacementPolicy. I am trying to get a list of all
DatanodeStorageInfo objects in order to be able to decide which one to
choose for placement (based on certain criteria), this is the logic of
chooseTarget method.
Could you please tell me how can I get this list? I tried to do the
following but I got none of these objects:
DistributedFileSystem fs = (DistributedFileSystem)FileSystem.get(conf);
//conf is passed in initialize method
List<DatanodeStorageInfo> storageInfos = new ArrayList<>();
for(DatanodeInfo node :
fs.getDataNodeStats(HdfsConstants.DatanodeReportType.LIVE)){
if(node instanceof DatanodeDescriptor){
storageInfos.addAll(Arrays.asList(((DatanodeDescriptor)node).getStorageInfo()));
}
It seems though that none of the resulting DatanodeInfo objects are
actually a DatanodeDescriptor therefor the storageInfos list is always
empty.
It would be great if you could provide me with more information on how to
get a list of DatanodeStorageInfo objects.
Thanks a lot in advance,
Youssef
