[jira] Created: (HDFS-1119) Refactor BlocksMap with GettableSet
Refactor BlocksMap with GettableSet --- Key: HDFS-1119 URL: https://issues.apache.org/jira/browse/HDFS-1119 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Reporter: Tsz Wo (Nicholas), SZE The data structure required in BlocksMap is a GettableSet. See also [this comment|https://issues.apache.org/jira/browse/HDFS-1114?focusedCommentId=12862118&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12862118]. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1120) Make DataNode's block-to-device placement policy pluggable
Make DataNode's block-to-device placement policy pluggable -- Key: HDFS-1120 URL: https://issues.apache.org/jira/browse/HDFS-1120 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Jeff Hammerbacher As discussed on the mailing list, as the number of disk drives per server increases, it would be useful to allow the DataNode's policy for new block placement to grow in sophistication from the current round-robin strategy. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1121) Allow HDFS client to measure distribution of blocks across devices for a specific DataNode
Allow HDFS client to measure distribution of blocks across devices for a specific DataNode -- Key: HDFS-1121 URL: https://issues.apache.org/jira/browse/HDFS-1121 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client Reporter: Jeff Hammerbacher As discussed on the mailing list, it would be useful if the DfsClient could measure the distribution of blocks across devices for an individual DataNode. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1122) client block verification may result in blocks in DataBlockScanner prematurely
client block verification may result in blocks in DataBlockScanner prematurely -- Key: HDFS-1122 URL: https://issues.apache.org/jira/browse/HDFS-1122 Project: Hadoop HDFS Issue Type: Sub-task Reporter: sam rash Assignee: sam rash found that when the DN uses client verification of a block that is open for writing, it will add it to the DataBlockScanner prematurely. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-760) "fs -put" fails if dfs.umask is set to 63
[
https://issues.apache.org/jira/browse/HDFS-760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan resolved HDFS-760.
--
Resolution: Fixed
This was fixed by HADOOP-6521.
> "fs -put" fails if dfs.umask is set to 63
> -
>
> Key: HDFS-760
> URL: https://issues.apache.org/jira/browse/HDFS-760
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 0.21.0
>Reporter: Tsz Wo (Nicholas), SZE
> Fix For: 0.21.0, 0.22.0
>
>
> Add the following to hdfs-site.conf
> {noformat}
>
> dfs.umask
> 63
>
> {noformat}
> Then run "hadoop fs -put"
> {noformat}
> -bash-3.1$ ./bin/hadoop fs -put README.txt r.txt
> 09/11/09 23:09:07 WARN conf.Configuration: mapred.task.id is deprecated.
> Instead, use mapreduce.task.attempt.id
> put: 63
> Usage: java FsShell [-put ... ]
> -bash-3.1$
> {noformat}
> Observed the above behavior in 0.21.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-808) Implement something like PAR2 support?
[ https://issues.apache.org/jira/browse/HDFS-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-808. --- Resolution: Duplicate > Implement something like PAR2 support? > -- > > Key: HDFS-808 > URL: https://issues.apache.org/jira/browse/HDFS-808 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Allen Wittenauer >Priority: Minor > > We really need an Idea issue type, because I'm not sure if this is really > viable. :) Just sort of thinking "out loud". > I was thinking about how file recovery works on services like Usenet to fix > data corruption when chunks of files are missing. I wonder how hard it would > be to implement something like PAR2 [ http://en.wikipedia.org/wiki/Parchive ] > automatically for large files. We'd have the advantage of being able to do > it in binary of course and could likely hide the details within HDFS itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1123) Need HDFS Protocol Specification
Need HDFS Protocol Specification Key: HDFS-1123 URL: https://issues.apache.org/jira/browse/HDFS-1123 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Reporter: bc Wong It'd be great to document (in a spec, not in the code) the HDFS wire protocol: * The layout of the different request and reply messages. * The semantics of the various calls. * The semantics of the various fields. For example, I stumbled upon the goldmine of comments around DataNode.java:1150. It looks correct, but the version number of 9 doesn't inspire confidence that it's up-to-date. (It's also a random place to put such an important comment.) Having a formal spec is a big step forward for compatibility. It also highlights design decisions and helps with protocol evolution. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
