[jira] Created: (HDFS-610) Add support for FileContext

2009-09-09 Thread Sanjay Radia (JIRA)
Add support for FileContext --- Key: HDFS-610 URL: https://issues.apache.org/jira/browse/HDFS-610 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client, name-node Affects Versions: 0.21.0

Re: Tracking Replication errors

2009-09-09 Thread Dhruba Borthakur
The sender datanode sends the crc along with the data. This allows the receiver datanode to detect corrupt data. The orignal crc was created by the client that created the data in the block for the first time. The crc is not kept in the namenode. To facilitate random access, there is a crc per 512

Re: Tracking Replication errors

2009-09-09 Thread Brian Bockelman
On Sep 9, 2009, at 10:25 PM, Dhruba Borthakur wrote: when a block is being received by a datanode (either because of a replication request or from a client write), the datanode verifies crc. Ah, so I'm wrong and the answer is better than I expected. Never have I been so happy to be wrong

Re: Tracking Replication errors

2009-09-09 Thread Dhruba Borthakur
when a block is being received by a datanode (either because of a replication request or from a client write), the datanode verifies crc. Also, the there is a thread in the datanode that periodically verifies crc of existing blocks. dhruba On Wed, Sep 9, 2009 at 7:27 PM, Brian Bockelman wrote:

Tracking Replication errors

2009-09-09 Thread Brian Bockelman
Hey everyone, We're going through a review of our usage of HDFS (it's a good thing! - we're trying to get "official"). One reviewer asked a good question that I don't know the answer too - could you help? To quote, "What steps do you take to ensure the block rebalancing produces non- cor

[jira] Created: (HDFS-609) Create a file with the append flag does not work in HDFS

2009-09-09 Thread Hairong Kuang (JIRA)
Create a file with the append flag does not work in HDFS Key: HDFS-609 URL: https://issues.apache.org/jira/browse/HDFS-609 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.

[jira] Created: (HDFS-608) BlockReceiver:receivePacket(): packet's header parsing logic is complicated. Refactoring will help w/ testing efforts

2009-09-09 Thread Konstantin Boudnik (JIRA)
BlockReceiver:receivePacket(): packet's header parsing logic is complicated. Refactoring will help w/ testing efforts - Key: HDFS-608 URL: https://

[jira] Resolved: (HDFS-603) Most replica related classes cannot be accessed

2009-09-09 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-603. - Resolution: Fixed I have committed this to the Append Branch. > Most replica relat

[jira] Created: (HDFS-607) HDFS should support SNMP

2009-09-09 Thread Allen Wittenauer (JIRA)
HDFS should support SNMP Key: HDFS-607 URL: https://issues.apache.org/jira/browse/HDFS-607 Project: Hadoop HDFS Issue Type: New Feature Reporter: Allen Wittenauer HDFS should provide key statistics over

Re: How to define the path of HDFS?

2009-09-09 Thread Boris Shkolnik
You should use FileSystem methods to do it.. E.g. FSDataInputStream open(Path f) On 9/8/09 9:54 PM, "Bill Yu" wrote: > Hi everyone, > > when I run the following code: > > ObjectInputStream in = new ObjectInputStream(new > FileInputStream("hdfs://localhost:9000/myDir/trajectory/tes

Re: How to define the path of HDFS?

2009-09-09 Thread Steve Loughran
Bill Yu wrote: Hi everyone, when I run the following code: ObjectInputStream in = new ObjectInputStream(new FileInputStream("hdfs://localhost:9000/myDir/trajectory/test.obj")); it throws an error that the directory is not exist, how can I define the its path ? Any suggestio