Eisuke Umeda created HDFS-10815:
-----------------------------------

             Summary: The state of the EC file is erroneously recognized when 
you restart the NameNode.
                 Key: HDFS-10815
                 URL: https://issues.apache.org/jira/browse/HDFS-10815
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha1
         Environment: 2 NameNodes, 5 DataNodes, Erasured code policy is set as 
"RS-DEFAULT-3-2-64k"
            Reporter: Eisuke Umeda


After carrying out an examination in the following procedures, an EC files came 
to be recognized as corrupt files.
These files were able to get in "hdfs dfs -get".
NameNode might be causing the false recognition.

NameNodes:
  namenode1(active)
  namenode2(standby)

The directory which there is "Under-erasure-coded block groups": 
/tmp/tpcds-generate/test
{code}
$ sudo -u hdfs hdfs erasurecode -getPolicy /tmp/tpcds-generate/test
ErasureCodingPolicy=[Name=RS-DEFAULT-3-2-64k, 
Schema=[ECSchema=[Codec=rs-default, numDataUnits=3, numParityUnits=2]], 
CellSize=65536 ]
{code}

The following is the steps to reproduce:
1) hdfs dfs -cp /tmp/tpcds-generate/25/store_sales/* /tmp/tpcds-generate/test
2) datanode1: (in the middle of the copy) sudo pkill -9 -f datanode
3) start a process of datanode1 two minutes later
4) carry out hdfs fsck and confirm that Under-Replicated Blocks occurred
5) wait until Under-Replicated Blocks becomes 0
6) (namenode1) /etc/init.d/hadoop-hdfs-namenode restart
7) (namenode2) /etc/init.d/hadoop-hdfs-namenode restart



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Reply via email to