[jira] Created: (HDFS-917) Write pipeline heartbeat interval should be determined by client timeout, not DN

2010-01-22 Thread Todd Lipcon (JIRA)
Write pipeline heartbeat interval should be determined by client timeout, not DN Key: HDFS-917 URL: https://issues.apache.org/jira/browse/HDFS-917 Project: Hadoop HDFS

[jira] Created: (HDFS-916) Rewrite DFSOutputStream to use a single thread with NIO

2010-01-22 Thread Todd Lipcon (JIRA)
Rewrite DFSOutputStream to use a single thread with NIO --- Key: HDFS-916 URL: https://issues.apache.org/jira/browse/HDFS-916 Project: Hadoop HDFS Issue Type: Improvement Componen

[jira] Created: (HDFS-915) Hung DN stalls write pipeline for far longer than its timeout

2010-01-22 Thread Todd Lipcon (JIRA)
Hung DN stalls write pipeline for far longer than its timeout - Key: HDFS-915 URL: https://issues.apache.org/jira/browse/HDFS-915 Project: Hadoop HDFS Issue Type: Bug Comp

[jira] Created: (HDFS-914) Refactor DFSOutputStream and DFSInputStream out of DFSClient

2010-01-22 Thread Todd Lipcon (JIRA)
Refactor DFSOutputStream and DFSInputStream out of DFSClient Key: HDFS-914 URL: https://issues.apache.org/jira/browse/HDFS-914 Project: Hadoop HDFS Issue Type: Improvement

[jira] Created: (HDFS-913) TestRename won't run automatically from 'run-test-hdfs-faul-inject' target

2010-01-22 Thread Konstantin Boudnik (JIRA)
TestRename won't run automatically from 'run-test-hdfs-faul-inject' target -- Key: HDFS-913 URL: https://issues.apache.org/jira/browse/HDFS-913 Project: Hadoop HDFS Issu

[jira] Created: (HDFS-912) sed in build.xml fails

2010-01-22 Thread Allen Wittenauer (JIRA)
sed in build.xml fails --- Key: HDFS-912 URL: https://issues.apache.org/jira/browse/HDFS-912 Project: Hadoop HDFS Issue Type: Bug Environment: ant 1.7.1 Solaris Reporter: Allen Wittenauer

[jira] Created: (HDFS-911) test-cactus fails with timeout error on trunk

2010-01-22 Thread Jakob Homan (JIRA)
test-cactus fails with timeout error on trunk - Key: HDFS-911 URL: https://issues.apache.org/jira/browse/HDFS-911 Project: Hadoop HDFS Issue Type: Bug Components: contrib/hdfsproxy

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Cosmin Lehene
Steve, A DoS could not be done using excludedNodes. The blacklisting takes place only at DFSClientLevel. The NN will return a list of block locations that excludes the nodes the client decided. This list isn't persisted anywhere on the server. So if a client excludes the entire set of DNs other

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Hairong Kuang
+1 Hairong On Fri, Jan 22, 2010 at 10:51 AM, Dhruba Borthakur wrote: > +1 for making this patch go into 0.21. > > thanks, > dhruba > > On Fri, Jan 22, 2010 at 10:25 AM, Todd Lipcon wrote: > > > Hi Steve, > > > > All of the below may be good ideas, but I don't think they're relevant to > > the

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Dhruba Borthakur
+1 for making this patch go into 0.21. thanks, dhruba On Fri, Jan 22, 2010 at 10:25 AM, Todd Lipcon wrote: > Hi Steve, > > All of the below may be good ideas, but I don't think they're relevant to > the discussion at hand. Specifically, none of them can enter 0.21 without a > vote as they'd be

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Todd Lipcon
Hi Steve, All of the below may be good ideas, but I don't think they're relevant to the discussion at hand. Specifically, none of them can enter 0.21 without a vote as they'd be new features, and it doesn't even sound like there's a JIRA out for them yet. Let's not put off a well-known improvement

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Dhruba Borthakur
> tweaked Hadoop to allow the datanodes to get the entire list are you referring to datanodes or dfs clients here? The client already gets the entire list of replica locations for a block from the namenode. and one could always develop a DFS client that is free to choose whatever locations it dec

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Steve Loughran
Stack wrote: I'm being 0 on this -I would worry if the exclusion list was used by the NN to do its blacklisting, I'm glad to see this isn't happening. Yes, you could pick up datanode failure faster, but you would also be vulnerable to a user doing a DoS against the cluster by reporting every

Re: [VOTE -- Round 2] Commit hdfs-630 to 0.21?

2010-01-22 Thread Cosmin Lehene
+1 On 1/22/10 3:24 AM, "Andrew Purtell" wrote: > +1 > > This makes an observed big difference for stability of small/test clusters. > > I second Ryan's specific point about stability of small clusters being > important. > > - Andy > > > On Thu Jan 21st, 2010 2:46 PM PST Ryan Rawson wro

[jira] Reopened: (HDFS-877) Client-driven block verification not functioning

2010-01-22 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened HDFS-877: -- This bug fix should go in 0.21 as well, I think. The current patch doesn't apply, but I'll try to get to one