Eisuke Umeda created HDFS-10815: ----------------------------------- Summary: The state of the EC file is erroneously recognized when you restart the NameNode. Key: HDFS-10815 URL: https://issues.apache.org/jira/browse/HDFS-10815 Project: Hadoop HDFS Issue Type: Bug Components: erasure-coding Affects Versions: 3.0.0-alpha1 Environment: 2 NameNodes, 5 DataNodes, Erasured code policy is set as "RS-DEFAULT-3-2-64k" Reporter: Eisuke Umeda
After carrying out an examination in the following procedures, an EC files came to be recognized as corrupt files. These files were able to get in "hdfs dfs -get". NameNode might be causing the false recognition. NameNodes: namenode1(active) namenode2(standby) The directory which there is "Under-erasure-coded block groups": /tmp/tpcds-generate/test {code} $ sudo -u hdfs hdfs erasurecode -getPolicy /tmp/tpcds-generate/test ErasureCodingPolicy=[Name=RS-DEFAULT-3-2-64k, Schema=[ECSchema=[Codec=rs-default, numDataUnits=3, numParityUnits=2]], CellSize=65536 ] {code} The following is the steps to reproduce: 1) hdfs dfs -cp /tmp/tpcds-generate/25/store_sales/* /tmp/tpcds-generate/test 2) datanode1: (in the middle of the copy) sudo pkill -9 -f datanode 3) start a process of datanode1 two minutes later 4) carry out hdfs fsck and confirm that Under-Replicated Blocks occurred 5) wait until Under-Replicated Blocks becomes 0 6) (namenode1) /etc/init.d/hadoop-hdfs-namenode restart 7) (namenode2) /etc/init.d/hadoop-hdfs-namenode restart -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org