[jira] [Created] (HDFS-13882) Change dfs.client.block.write.locateFollowingBlock.retries default from 5 to 10
Kitti Nanasi created HDFS-13882: --- Summary: Change dfs.client.block.write.locateFollowingBlock.retries default from 5 to 10 Key: HDFS-13882 URL: https://issues.apache.org/jira/browse/HDFS-13882 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 3.1.0 Reporter: Kitti Nanasi Assignee: Kitti Nanasi More and more we are seeing cases where customers are running into the java io exception "Unable to close file because the last block does not have enough number of replicas" on client file closure. The common workaround is to increase dfs.client.block.write.locateFollowingBlock.retries from 5 to 10. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[Discuss] Merge discussion for Node attribute support feature YARN-3409
Hi All, We would like to hear your thoughts on merging “Node Attributes Support in YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to get it in for HADOOP 3.2. *Major work happened in this branch* YARN-6858. Attribute Manager to store and provide node attributes in RM YARN-7871. Support Node attributes reporting from NM to RM( distributed node attributes) YARN-7863. Modify placement constraints to support node attributes YARN-7875. Node Attribute store for storing and recovering attributes *Detailed Design:* Please refer [1] for detailed design document. *Testing Efforts:* We did detailed tests for the feature in the last few weeks. This feature will be enabled only when Node Attributes constraints are specified through SchedulingRequest from AM. Manager implementation will help to store and recover Node Attributes. This works with existing placement constraints. *Regarding to API stability:* All newly added @Public APIs are @Unstable. Documentation jira [3] could help to provide detailed configuration details. This feature works from end-to-end and we tested this in our local cluster. Branch code is run against trunk and tracked via [4]. We would love to get your thoughts before opening a voting thread. Special thanks to a team of folks who worked hard and contributed towards this efforts including design discussion / patch / reviews, etc.: Weiwei Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong Chen. [1] : https://issues.apache.org/jira/secure/attachment/12937633/Node-Attributes-Requirements-Design-doc_v2.pdf [2] : https://issues.apache.org/jira/browse/YARN-3409 [3] : https://issues.apache.org/jira/browse/YARN-7865 [4] : https://issues.apache.org/jira/browse/YARN-8718 Thanks, + Naga & Sunil Govindan
[jira] [Resolved] (HDDS-132) Update SCMNodeStorageStatMap while processing Node Report
[ https://issues.apache.org/jira/browse/HDDS-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDDS-132. --- Resolution: Implemented > Update SCMNodeStorageStatMap while processing Node Report > - > > Key: HDDS-132 > URL: https://issues.apache.org/jira/browse/HDDS-132 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Blocker > Fix For: 0.3.0 > > > When the node report is received at SCM, SCMNodeStorageStatMap needs to get > updated. > In the event of a node/Volume failure, this Map needs to be updated as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Resolved] (HDDS-108) Update Node2ContainerMap while processing container reports
[ https://issues.apache.org/jira/browse/HDDS-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDDS-108. --- Resolution: Implemented > Update Node2ContainerMap while processing container reports > --- > > Key: HDDS-108 > URL: https://issues.apache.org/jira/browse/HDDS-108 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-108.00.patch, HDDS-108.01.patch, HDDS-108.02.patch, > HDDS-108.03.patch, HDDS-108.04.patch > > > When the container report comes, the Node2Container Map should be updated via > SCMContainerManager. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
