Hi,
I am trying to create a file in hdfs . I am calling create from an
instance of DFSClient. This is a part of code that i am using
byte[] buf = new byte[65536];
int len;
while ((len = dis.available()) != 0) {
if (len < buf.length) {
break;
} else {
dis.read(buf, 0, buf.length);
ds.write(buf, 0, buf.length);
}
}
dis is DataInputStream for the local file system from which i am copying
the file and ds is the DataOutputStream to hdfs.
and i get these errors.
2010-07-03 13:45:07,480 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode:
DatanodeRegistration(127.0.0.1:50010,
storageID=DS-455297472-127.0.0.1-50010-1278144155322, infoPort=50075,
ipcPort=50020):DataXceiver
java.io.EOFException: while trying to read 65557 bytes
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.readToBuf(BlockReceiver.java:265)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.readNextPacket(BlockReceiver.java:309)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:373)
at
org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:525)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:357)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:103)
at java.lang.Thread.run(Thread.java:636)
When i run the loop for number of times that is a multiple of block size ,
the operation runs just fine. As soon as i change the buffer array size to
a non block size , it starts giving errors.
I am in middle of a project . Any help will be appreciated.
thanks
vidur
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
